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ABSTRACT 


Each graduating class from the United States Naval Academy experiences an 
attrition rate of approximately 24 percent. A significant portion of that attrition is 
attributed to academic difficulties. The Academy provides various programs such as the 
Plebe Intervention Program (PIP) to assist midshipmen experiencing academic difficulty. 
The purpose of this study was to develop an empirical approach to selecting first-year 
Naval Academy Midshipmen for academic intervention based upon objective initial entry 
data. Categorical values from the Learning and Study Strategies Inventory (LASSD, 
SAT scores and high school rank were incorporated as independent variables in a linear 
regression model with dependent variable Cumulative Quality Point Rating (CQPR). 
Two regression analyses were conducted to develop the final equation. Results of the 2" 
regression indicate class standing, individualized SAT Math and SAT Verbal scores were 
highly significant at the 0.01 level relative to academic performance. Several LASSI 
variables also proved valuable in the model. Motivation, Attitude, Time Management, 
Select Main Ideas, Study Aids and Test Strategies were also significant at the 0.05 level. 
While these results were encouraging, the low R’ value of 3.27 indicated that the model 
could not predict CQPR for a specific case with accuracy. However, utilizing this 
equation empirically enhances current intervention program selection processes 


significantly improving the identification of academically challenged midshipmen. 
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I. INTRODUCTION 
A. BACKGROUND 

Academic attrition has been a subject of concern for years in both civilian and 
military institutions of learning. (Lavin, 1965; Marcus, 1989; Schreiner, 1988; Congos, 
1997) In order to stay viable, colleges, universities and military academies alike, must be 
able to recruit and retain enough students to meet the operating goals of the institution. 
While the academic and operational goals for civilian and military institutions may differ, 
the underlying requirement of maintaining a sufficient student body remains. At highly 
selective institutions, the challenge is even more daunting. 

Student attrition can have significant effects throughout an institution. In civilian 
schools, the student body brings with it major sources of funding in the form of tuition, 
grants, scholarships, dormitory fees, and support of various other services provided by 
the institution. These sources of funding help to pay for faculty, service and support 
staff, facility maintenance and development costs, utilities and many other administrative 
and operational costs associated with the institution. Therefore, if the school is not 
attracting and retaining enough students (i.e. sources of income) to help balance its 
operating costs, then the institution will not remain viable and could become the subject 
of unsolicited attention. Military institutions must deal with similar social and financial 
issues as well as political considerations. This topic will be addressed further in later 
discussion. 

Another factor of student attrition is its effect on the reputation of an institution. 
Most institutions pride themselves on providing an academically challenging 


environment for their students. The problems of adjusting to this challenging 
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environment while coping with the reality of being away from home can be significant 
impediments to student retention. If the challenges provided by the institution are not 
tempered with sound judgment and realistic goals, then one measure of institutional 
reputation may be a high attrition rate for the school or a particular program. In civilian 
and military institutions this is a double-edged sword because a reputation of 
disproportionately high attrition, may turn away prospective students who would 
otherwise be interested in applying for admission. It is important for an institution to 
monitor student attrition and make every attempt to understand its root causes. 

Military institutions must also focus on student attrition but for different reasons. 
The primary sources of funding for military institutions such as the United States Naval 
Academy originate not from individual students but from government resources. The 
U.S. Naval Academy serves as a major source of officers for the Navy and Marine Corps 
and its sole purpose is to produce officers who will lead the men and women of our 
armed forces to support and defend the United States against all enemies foreign and 
domestic. 

Every taxpaying United States citizen is a stakeholder in the final product of the 
Naval Academy. Therefore, the Academy must ensure that it is doing everything 
possible to prepare midshipmen to be effective and successful in military service. The 
public nature of the Academy requires that close attention be paid to maintaining the 
quality and necessary quantity of its products. Similar to civilian institutions, attrition at 
the Naval Academy directly affects public perception. For example, imagine that the 
Naval Academy’s Class of 2030 inducted twelve hundred midshipmen for plebe summer, 


but only graduated 600 midshipmen four years later. An attrition rate of this magnitude 
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would indicate inefficiency and ineffectiveness within the organization. While this is an 
extreme example, it illustrates the importance of managing attrition effectively. A certain 
degree of attrition is appropriate and natural. Not every individual admitted to the Naval 
Academy possesses the basic but elusive building blocks from which military leaders are 
developed. The current attrition rate, released by the Naval Academy Institutional 
Research Department is approximately twenty-four percent. This means that of the 
twelve hundred midshipmen admitted to the academy each year, approximately three 
hundred do not make it to graduation. The Office of the Superintendent, US Naval 
Academy estimates the cost of educating and training a midshipman at over two hundred 
thousand dollars. Therefore, it is important that administrators seek to minimize attrition 
where possible. Is it possible and practical for the Naval Academy to reduce the current 
attrition rate? This study focuses on one aspect of this question. 

One of several reasons for the current attrition rate is academic deficiency. 
Organizations such as the Naval Academy Academic Center have been established in an 
attempt to assist those midshipmen who encounter academic difficulties in order to 
provide them with the basic tools they need to succeed academically. There are several 
other contributors to the attrition rate. Some of them include: conduct and honor 
offenses, physical deficiencies or injury, and professional performance. All of these 
forms of attrition affect the academy, however this study focuses on the prediction of 
academic performance in the hopes of developing an empirical approach to assist in the 
selection of midshipmen for academic assistance programs. 

Academic performance is extremely important at the Naval Academy. A college 


education is a prerequisite for most officers with very few exceptions. While the Naval 
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Academy provides the opportunity for midshipmen to pursue very diverse fields of study, 
the core required courses provide every graduate with a rigorous background in 
engineering. Regardless of major selection, every midshipman maintains a very busy and 
rigorous academic schedule. Academics can also affect various aspects of midshipman 
life. Privileges such as liberty and weekends, leadership opportunity within the brigade 
of midshipmen, overall class standing and, most importantly, service assignment are 
dependent, in varying degrees, upon academic performance. Academic standing also 
plays into lineal number generation. An individual’s lineal number determines where a 
class member falls out with respect to peers from all accession sources within a particular 
year group. This number follows an academy graduate throughout their naval career and 
is a seniority factor that can affect promotion and command opportunities. Obviously, 
academic performance weighs heavily on every midshipman as they progress through the 
Naval Academy. 

The Naval Academy requires that every graduate maintain a 2.0 overall 
Cumulative Quality Point Rating (CQPR) in order to graduate from the Naval Academy. 
Midshipmen failing to meet the academic criteria can be subject to review by an 
academic board and subsequently separated from the Naval Academy. As previously 
stated, the Naval Academy loses approximately 24 percent of each entering class before 
graduation. While some attrition is expected and planned for each year and the academy 
traditionally supplies the Navy with a sufficient corps of officers, it is important to 
minimize unnecessary losses simply because the costs involved are too high. It costs 
over $200,000 to produce one officer for the Navy or Marine Corps. To lose a 


midshipman in their third or forth year for any reason is somewhat recouped by either a 
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partial monetary repayment or enlisted service obligation, but the tradeoff is significant. 
By the time a midshipman has reached their Second Class (junior) year, a significant 
amount of funding has already been lost to summer training, food, housing, travel, pay 
and educational expenses. If a midshipman is lost prior to the third academic year, then 
all funds consumed are lost because there is no obligation or payback from the 
midshipman. They simply leave with two years of fully funded military training and 
college education under their belt. Therefore by attempting to understand and assess 
academic potential, administrators could focus on midshipman who may otherwise 
struggle academically. It is reasonable to say that most midshipmen want to succeed 
academically. Given the right tools and motivation, those who may struggle could 
benefit from assistance programs and theoretically successfully graduate given that they 
encounter no other problems associated with attrition. 

Since academic performance can influence attrition, it is incumbent upon Naval 
Academy administrators to provide services that “at-risk” midshipmen may utilize to 
improve their academic skills and subsequently reduce chances of academic attrition. 
Some may argue that we should simply screen out those who may be academically “at- 
risk”. In response, the argument could also be made that for any class of midshipmen 
entering the Academy; there will always be midshipmen who exhibit a lower academic 
performance level than their peers. However, the purpose of the Naval Academy is to 
provide good officers and there is no historical evidence to suggest that a poor student 
cannot be an excellent officer. 

Identification of “at-risk” midshipmen requires study of historical data to estimate 


significant factors affecting academic performance. The United States Naval Academy 
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Academic Center was created to deal specifically with midshipmen having academic 
difficulties. Voluntary referral to the Academic Center is welcomed, however resources 
are limited so it is important to identify and attempt to provide services to those 
midshipmen most in need of assistance. 

Generally, high school rank, SAT scores and other objective measurable criteria 
are used to determine one’s academic potential. Studies show that these performance 
characteristics are indeed significant in relation to academic performance. (Astin, 1968; 
Congos, 1997; Lavin, 1965) However, there are other subjective factors that may also 
significantly affect academic performance. Lee 1970 discusses specific interest factors 
that influence different areas of study such as biology, chemistry and mathematics. It 
was found that interest characteristics and personality traits could either positively or 
negatively affect academic performance for specific areas of study. At the Naval 
Academy, Plebe Midshipmen are administered the Learning and Study Strategies 
Inventory (LASSI) prior to starting academic studies. This inventory consists of 77 
questions relating to 10 areas of learning and study ability. Some or all of these areas 
may significantly affect academic performance. This study attempts to assess whether 
these 10 variables along with other objective criteria can be used to develop a relationship 


that effectively predicts academic performance. 


B. PURPOSE 


The overall objective of this thesis is to research and develop a relationship that 
can be used by the Academic Center to select midshipmen for academic intervention. By 


identifying independent variables that significantly affect academic performance, the 


USNA Academic Center could identify and predict academic performance for incoming 
midshipmen. The predicted performance characteristics could be implemented as a 


measure for selecting midshipmen in need of assistance. 


C. RESEARCH QUESTIONS 
1. What entry-level independent variables significantly affect Cumulative Quality 
Point Rating (CQPR)? 
2. Is the Learning and Study Strategy Inventory (LASSD) an indicator of academic 
performance? 
3. Can an empirical model be developed to select midshipmen for academic 
intervention? 
4. Should the LASSI be administered to all incoming midshipmen? 
5. When should the LASSI be administered? 
6. Can a prediction model be implemented to assist in the selection of 


midshipmen for academic assistance? 


D. BENEFITS OF STUDY 

This study will provide an empirical process for selecting midshipmen for 
academic intervention based upon historical data. Since academic grades directly affect 
attrition as well as class standing, it is hoped that by defining academic performance 
based upon collected entry level data the selection process can be improved to identify 


midshipmen genuinely in need of assistance. 


E. SCOPE, LIMITATIONS AND ASSUMPTIONS 

1. Scope 

The main focus of this study was to determine whether it is possible to predict 
academic performance during plebe year. Data for seven successive Naval Academy 
classes from year groups 1996 through 2003 was collected. Specific content and 
discussion of data is included in the literature review section of this study. 

Every midshipman graduates from the Naval Academy with a Bachelor of 
Science degree and most take the same or similar classes their plebe year. However, 
once midshipmen select an academic major, departmental curriculum differences 
preclude academic comparisons for this particular study. Lavin, 1965 addresses this 
specific problem noting that uncontrolled sources of variation in grades obtained from 
students taking different courses of instruction introduces a fundamental error into the 
prediction of overall grade point average. Therefore, only plebe year performance 
characteristics were analyzed for each class year in order to provide a more uniform 
academic profile from which to make comparisons and base predictions. 

The remaining scope of the thesis consists of statistical analysis of data collected 
for each class using linear regression and finally a discussion of results observed with 


recommendations for future research. 


2. Limitations and Assumptions 
There are some inherent limitations to conducting a quantitative analysis within 
this field of study. First, this study attempts to implement objective and subjective 


criteria to determine a measurable quantity. Factors such as motivation and desire are 


difficult to quantify and usually take the form of simple estimations based upon observer 
or interviewer perception of an individual. This type of analysis is fundamentally flawed 
because one interviewer’s perception of “motivation” could be very different from 
another interviewer’s perception and result in a large degree of error. Other social or 
external influences will be difficult to take into account. One example of social 
influence is family pressure. While the results of this pressure may be evident in some 
areas, the full effects on specific performance areas may be elusive. 

Use of the LASSI within the analysis attempts to quantify some subjective factors 
that may affect academic performance. However, it is possible that some unknown or 
undetermined subjective factors will be neglected and will consequently fall out as error 
in the analysis. The key is to minimize this error term to provide the most accurate 
relationship possible. For the purposes of this study, it was assumed that every 
midshipman entering the academy desired and put forth the effort to succeed at the 
academy where success is defined as graduation from the institution. It was also assumed 
that midshipmen entering the PIP were motivated to apply program techniques and 
guidance to their studies. These assumptions are critical in allowing for clear 
interpretation of collected data but should be kept in mind for error considerations. 

Second, there is bound to be a degree of correlation among some of the 
independent variables such as high school grades and SAT scores. Simply put, this 
means that some independent variables may have similar effects on academic 
performance because they are directly or indirectly related to each other as well as the 
dependent variable. For example, high school grades may be a positive indicator of SAT 


scores as well as an indicator of academic performance. Therefore high school grades 
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and SAT scores are related to each other and any effects on a third dependent wariable 
will have to take this into account. It is important to understand, define, and account for 
these relationships to provide the best possible prediction. While it may be fairly simple 
to determine correlations among variables that are closely related, it may be more 
difficult to identify subtle relationships among variables. Neglecting these subtle 
relationships may decrease the effectiveness of the model equation in predicting the 
dependent variable. It was assumed that, by definition, the hypothesized model will not 
ultimately define academic performance and that there will be an error term involved. 
One final limitation is that some of the raw LASSI data for certain classes is 
missing and will prevent use of those midshipmen in the data analysis regarding LASSI 
influence. Specifically, LASSI data for the classes of 1998 — 2000 is unavailable due to 
technical difficulties in the transfer of data during shutdown of the Naval Academy 
Timesharing System (NATS). This system held all LASSI information and when it was 
disabled, some of that data was lost. Since there is still a significant amount of data 
available to complete the study this will not affect the overall analysis, but subtleties 
among classes may be missed due to this omission. As part of the data analysis section, a 
comparison will be made between data collected before this three-year gap and data 


obtained afterward. 


F. ORGANIZATION OF STUD Y 


This study consists of six chapters: introduction, literature review, data 
description and methodology, hypothesis, results, and conclusions and recommendations. 


The literature review includes a brief introduction, an overview of all materials 
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referenced, discussion of the LASSI, discussion of the Supplemental Instruction (SI) 
program, model justification and discussion, and a final chapter summary. The data 
description and methodology section includes a brief introduction, discussion of the 
variables to be analyzed, visual representation and discussion of the model equation, 
cross-tabulations, descriptive statistics and a chapter summary. The hypothesis chapter 
consists of a basic introduction, model discussion and representations and a chapter 
summary. The results section consists of all regression data including charts, models, 
regression tables, coefficients and discussion. Finally, the last chapter consists of all 


conclusions and recommendations for further research. 
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II. LITERATURE REVIEW 
A. INTRODUCTION 
The body of literature reviewed for this study covers various aspects of academic 
performance and assistance. Personality characteristics, objective measurable 
achievement, and past performance levels have been the subject of studies for decades. 
The purpose of this chapter is to review some of the existing research and develop the 


baseline from which this study was launched. 


B. NAVAL ACADEMY ACADEMIC CENTER 

“The Academic Center exists to support the mission of the Naval academy by 
providing the highest quality academic support programs for midshipmen seeking to 
improve their academic performance.” (Bowman, 1998) The Naval Academy Academic 
Center was established in 1989 and strives to achieve this mission each academic year by 
providing various services to midshipmen primarily on a voluntary basis. These services 
include plebe programs, academic counseling, learning skills and tutoring. The goals of 
these programs are twofold: “To provide excellent academic support services to enable 
midshipmen to work to their full potential in a rigorous educational environment and to 
teach basic learning skills necessary for effective academic performance.” (Bowman, 


1998) This study focuses on the Plebe Programs area of the Academic Center. 


1. Plebe Programs 
There are two primary ervices provided within Plebe Programs. The Plebe 


Intervention and Plebe Advising programs are both designed to help incoming 
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midshipmen adjust to the rigorous academic environment of the Naval Academy by 
providing various forms of instruction and counseling. The purpose of this intervention 
is to develop or improve existing academic skills in the hopes of providing a skill set the 
learner can implement successfully in and out of the classroom. This study focuses 
primarily on the Plebe Intervention Program. 

The Plebe Intervention Program (PIP) provides proactive support to midshipmen 
identified as “at-risk” by the program director, athletic coaches, company officers, 
academic boards and other sources. Midshipmen identified as program participants are 
paired with an academic advisor and provided counseling and learning skills instruction 
as well as weekly advising meetings with the program administrators throughout the 
academic year. Participation is primarily voluntary although some midshipmen may 
receive directives to participate from the academic board or their company officers. 
Plebe Intervention is designed to maximize the potential of midshipmen who may 


otherwise have significant difficulties performing academically. 


2. Selection Process 


Approximately 100 midshipmen per year are offered the opportunity to participate 
in the Plebe Intervention Program. Typically, the bulk of these midshipmen are 
identified and notified during plebe summer based upon initial entry information such as 
admissions data and board feedback, Naval Academy Athletic Association (NAAA) 
recommendations, and Naval Academy Preparatory School (NAPS) data. These sources 
are utilized to prepare a watch list of 200 midshipmen. Subsequently, placement exams 


for all of the midshipmen on the watch list are reviewed and approximately 60 are 
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selected for enrollment in the PIP. The remaining 140 are monitored for the first six 
weeks of the academic semester. At the time of this study, this program had the capacity 
to service approximately 100 midshipmen. Therefore, in the weeks following the first 
six-week grading period, the remaining 40 midshipmen are selected from the watch list 


for inclusion in the program. 


3. Intervention 


Midshipmen who are selected or volunteer to participate in the Plebe Intervention 
Program are counseled on a weekly basis and are provided with individualized and group 
study skills instruction. The purpose of this instruction is to give each midshipman the 
tools required to build effective study habits and assist them in preparing for the rigorous 
academic environment of the United States Naval Academy. Some midshipmen may 
even be placed in basic- level courses designed to teach the fundamentals of a subject who 
exhibits a below average level of understanding of the course material. This program is 
not designed to exist in lieu of extra instruction with professors and students are expected 
to interact with their individual instructors as well as the Academic Center when 


problems are encountered. 


C. LASSI 


The Learning and Study Strategies Inventory (LASSI) was developed at the 
University of Texas at Austin as part of a cognitive learning strategies project in 1978. 
The primary purpose of this cognitive study was to address growing concerns about 
under prepared students entering post-secondary educational environments. It was hoped 


that findings could be developed into an assessment tool to identify students’ academic 
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strengths and weaknesses. This would prove extremely useful in developing databases 
for trend analysis and designing ability specific training programs for those in need of 
academic assistance. (Weinstein, 1987). 

The LASSI was developed over several years through data gathering, expert 
consultation and statistical analysis. A panel of experts utilized all available data to 
assemble 645 items that were determined could possibly relate to learning. This pool was 
then reduced to 291 by eliminating items that were related to personality characteristics, 
were poorly worded or duplicated, and those that did not directly relate to study practices. 
Next a series of pilot tests were conducted over several years with results being compared 
to similar proven test procedures, standardized test correlations and academic 
achievement results. After adjusting to these criteria, various items were added to and 
deleted from the survey in its final form. The resulting LASSI consists of a 77-item 
questionnaire in Likert format. It has been tested and validated in its current form by 
over 30 colleges and universities. In all cases few difficulties were reported and all 
reported a high degree of usefulness. 

LASSI results are computed by each student following administration of the 
survey and are presented in percentile format. There are a total of ten categories 
assessed; attitude, motivation, time management, anxiety, concentration, information 
processing, selecting main ideas, study aids, self testing, and test strategies. Each student 
can then compare their performance with national norms provided within the survey 
packet. Average performance for each category was determined to be between the 50" 
and 75'" percentiles. This information has proved useful to administrators in identifying 


students who may need additional assistance in specific areas. 


16 


At the United States Naval Academy, the LASSI is administered prior to 
commencement of academic studies during plebe year. The survey is then scored, 
reviewed and stored at the Naval Academy Academic Center for future reference. It is 
currently considered as a minor factor in selecting midshipmen for the Plebe Intervention 
Program. This study seeks to identify whether individual categories of the LASSI, along 
with other objective criteria, may be used to specifically identify midshipmen for the 
Plebe Intervention Program. A review of each category and further discussion of the 


LASSI survey is included in subsequent chapters. 


D. SUPPLEMENTAL INSTRUCTION 


Over the years, there have been many approaches to academic assistance in 
college. One of the most prominent and researched has been a voluntary assistance 
program called Supplemental Instruction (SI) was founded at the University of Missourt+ 
Kansas City. It was in founded in 1973 by Deanna C. Martin for voluntary use of 
students enrolled in various professional studies including medicine, dentistry and 
pharmacy. Its purpose was to provide a northreatening learning environment outside of 
normal class hours for specific high-risk courses of instruction. Discussion of the 
Supplemental Instruction program is valid for the purposes of this thesis because the 
volume of research detailing positive results observed by students participating in 
Supplemental Instruction provides a basic premise that assistance programs can work if 
properly implemented. 

As previously stated, this program focuses on high-risk courses. For a course to 


be identified as high-risk, 30 percent of all final grades for the semester had to be D’s, F’s 


id, 


or withdrawals. The fact that specific classes were identified as high-risk is interesting in 
itself because the term “high-risk” is typically associated with student potential not 
course difficulty. Martin (1993) quotes Laura Rendon from the May 1989 edition of the 
American Association of Higher Education Bulletin: 

Our educational institutions, particularly higher education, like to perceive 

themselves as pillars of perfection. When something goes wrong with the 

system, it is easier to blame the victim for contaminating the system. In so 

doing, institutions practice scapegoating and focus on the needs or 

deficiencies of students instead of facing up to the institutions own 

imperfections. (page 1.) 

At the time, this was a fundamental shift in learning theory. In the traditional 
sense, student academic potential is assessed using measures such as SAT and ACT 
scores, prior academic performance, and certain subjective factors such as essays, 
interviews and recommendations. Supplemental Instruction focuses on the courses, not 
the students and establishes a forum through which students enable themselves to learn. 
This idea was formulated because of rising concerns about college attrition in recent 
years. “Tinto (1987) predicted in 1986 that, of the nearly 2.8 million students who 
entered higher education for the first time, over 1.8 million would leave without receiving 
a degree.” (Martin, 1993) Tinto went on to identify four significant elements of student 
attrition: social isolation, difficulty adjusting, linking class material to prior knowledge 
and difficulty in the college environment. Supplemental Instruction was developed as an 
institutional response to these four obstacles to learning. There are several benefits to 
the Supplemental Instruction program. First, it is proactive rather than reactive. Students 


sign up for the program before classes begin and are recommended for classes with 


Supplemental Instruction sessions based upon objective entry criteria. Supplemental 
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Instruction is implemented in courses that students have the most difficulty with, but it is 
course specific. A Supplemental Instruction session in history will be very different from 
a Supplemental Instruction session in Chemistry. They are geared toward the course 
material, not specific study skills. Supplemental Instruction leaders attend every class 
session so as to better facilitate discussion of difficult material. Supplemental Instruction 
is not considered a remedial program even though it is designed to improve student 
performance. Students who participate represent the entire range of academic 
performance levels. One last benefit is that Supplemental Instruction sessions are 
designed to promote interaction and cooperation whereby students work in groups to 
mutually support each other. 

There are three key personnel categories that oversee the implementation and 
administration of Supplemental Instruction sessions. The Supplemental Instruction 
Leader is a student who acts as a learning facilitator. Typically, the leader has taken the 
subject course or a comparable course previously, attends all classes, takes notes and 
reads all course assignments. In essence, the leader is a model student who guides the 
group in mutual learning. The Supplemental Instruction Supervisor oversees the entire 
Supplemental Instruction program, identifying high-risk courses, supervising 
Supplemental Instruction Leaders and gaining faculty support. Faculty members make 
up the remaining group. They screen Supplemental Instruction Leaders for competency 
and must be willing to participate in the program for it to be effective. In essence, 
Supplemental Instruction is a very simple concept relying on only a few dedicated key 


personnel to be effective. 
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While the academic assistance programs provided by the Naval Academy 
Academic Center are not Supplemental Instruction programs; there are several 
similarities between the two that lend to comparison. Naval Academy programs and 
Supplemental Instruction both focus on early intervention. While the Naval Academy 
gears toward study skill development, counseling and some peer assisted tutoring, 
Supplemental Instruction is strictly peer facilitated, but both primarily assemble prior to 
class convening. Supplemental Instruction is not a remedial program and neither is the 
Naval Academy Plebe Intervention Program. The Plebe Intervention Program does target 
midshipmen having potential for academic difficulty, however this does not mean that the 
program is a remedial tool. It simply means that midshipmen are targeted in the hopes of 
preventing the need for remedial measures once poor performance has been established. 
A similar selection process is performed by academic advisors promoting use of 
Supplemental Instruction programs. Students are screened prior to commencement of 
classes and recommended to sign up br classes with Supplemental Instruction sessions 
attached. As with the Naval Academy program, students are not required to participate in 
Supplemental Instruction sessions. One final similarity is that Supplemental Instruction 
sessions are designed to enable students through mutual support, cooperation and 
interaction. The Naval Academy programs are designed to enable students by developing 
tools they can use to enhance their educational atmosphere. These tools include study 
skills, group interaction, aademic counseling, and, if needed, individual and group 
tutoring in specific courses. It is important to realize that these two programs are not 
parallel in scope and operation, but the fact that both primarily seek to enhance and 


improve the learning environment through outside intervention leads to comparison. 
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Extensive research has shown Supplemental Instruction to be valid and successful. 
Martin, (1993) found that students participating in Supplemental Instruction earned a 
significantly larger percentage of A and B course grades and earned significantly less D, 
F, and withdrawal values. It is important to note here that Martin’s study controlled for 
the academic potential of students enrolling in Supplemental Instruction programs and 
that comparisons were made to motivated non-Supplemental Instruction students as well 
as all non Supplemental Instruction students. Motivation levels were established via 
survey data taken prior to commencement of academic classes. In all cases, students who 
participated in Supplemental Instruction programs outperformed those who did not 
participate. Specifically, 44.5 percent of all Supplemental Instruction students received 
final grades of A or B and 16.7 percent received grades of D, F, or W. In the 
motivational control group, only 34.5 percent received an A or B and 34.5 percent 
received a D, F or W. The difference was even more pronounced in the norm 
Supplemental Instruction-non- motivational group where 26.3 percent received an A or B 
and 51.1 percent received grades of D, F or W. 

It would not be appropriate to parallel this program directly with the Naval 
Academy’s program due to significant differences in implementation. However, the fact 
that Supplemental Instruction has been validated through extensive esearch (Martin, 
1993; Koechner, 1997) and exists as a student intervention program leads to the 
conclusion that intervention programs can and do improve student performance if 
properly implemented. One major goal of this study is to assist in the identification of 


midshipmen for participation in the Naval Academy Plebe Intervention Program in the 
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hopes of exercising early intervention for those who may experience academic 


difficulties. 


E. SIGNIFICANT VARIABLES 


Increasing interest in collegiate attrition rates has led the academic community to 
study factors that contribute to academic achievement. A wealth of research exists on the 
subject covering various objective and subjective variables. Most studies use some form 
of cumulative grade point average as a measure of overall student achievement. (Astin, 
1968; Blanc, 1983; Collins, 1982; Congos, 1997; Crawford, 1948; Koechner, 1997; 
Lavin, 1965; Lee, 1970; Weitzman, 1981) Using this measure as a dependent variable, 
many studies then develop a hypothesis as to what variables may help predict the 
dependent variable. These independent variables may be objective or subjective in nature 
and include items such as student survey data, entry variables such as SATs, ACTs, and 
high school rank, personality characteristics, intellectual factors, demographical data and 
participation in intervention programs. While there are differences in every study 
ranging from the data set utilized to variations in the type of independent variable and 
formatting used, similarities and trends have become apparent over the years. For 
example, several studies have noted that the Scholastic Aptitude Test and high school 
rank are fairly strong predictors of academic performance. (Collins, 1982; Weitzman, 
1981) 

Collins, (1982) studied the affects of high school rank, supplemental instruction 
and individualized math and verbal SAT scores upon overall grade point average and 


individual course grade. Results indicated that SAT math and high school rank were 
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powerful indicators of academic performance in math and science based courses such as 
biology and chemistry. SAT verbal scores were little use in predicting grades in 
technical classes, but were not evaluated relative to other nonmscience courses of 
instruction. Collins did note “utilization of the supplemental instruction program can 
make a significant contribution to the academic achievement of students.” (Collins, 1982) 
This infers the basic assumption that there is a positive correlation between academic 
intervention and academic performance. In Lavin, (1965) problems associated with the 
prediction of academic performance were discussed along with the idea that personality 
characteristics may have a significant impact upon academic potential. Specifically, 
several problems with prediction were noted. Students do not take the same courses with 
the same instructors. Not only are there very great differences in course material, there 
are certainly great differences in grading criteria from instructor to instructor. Therefore, 
Lavin observed that any study attempting to predict academic performance must take this 
into account by isolating certain aspects of academic study or discussing error statistics 
during the research process. This creates a significant boundary to research by restricting 
when and how much data can be utilized at any given point in time. The current study 
attempts to control for a good deal of inconsistency by focusing primarily upon the first 
academic year when the majority of the Naval Academy Plebe class takes the same, if not 
very similar courses. The first academic year for each incoming class provides the 
optimum time for researching academic performance across the brigade of midshipmen. 
After plebe year, selection of different majors and elective courses makes accurate study 
much more difficult. Lavin also discusses the potential for personality characteristics to 


affect academic performance. Motivation, attitude toward academic endeavors and 
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measures of independence were some of the characteristics that were observed to 
positively correlate with academic performance. Lavin summarizes several inventory- 
like studies that focus on these subjective factors with some interesting specific 
correlations. Anxiety was found to be a negative predictor of academic performance in a 
test-taking environment. Specifically, individuals that were highly permeable, or 
influenced by internal and external stimuli, and low on stability, or susceptible to nervous 
tension, would perform negatively in a high stress environment. This may seem to be an 
obvious assessment within the civilian academic environment. However, at the Naval 
Academy stress is an important element of the training process and midshipmen are 
forced to learn coping mechanisms and perform under stress to a much higher degree 
than is realized in civilian institutions. Therefore, anxiety may not play as critical a role 
in predicting academic performance as in other academic environments. In summary, 
Lavin’s study provides more fuel to support the idea that subjective personality factors 
can influence performance in the classroom. The key is to accurately identify which 
characteristics are most influential and then try to isolate ways to measure these 
characteristics for an individual. 

Conclusions summarized in Weitzman’s 1981 study support the view that high 
school record and SAT scores combined provide a very strong indicator of performance 
with no discussion of personality characteristics. “The predictive validity of 0.72 
indicates that the HSR-SAT (high school record-scholastic aptitude test) combination is a 
powerful predictor of first-year academic achievement in college.” (Weitzman, 1981) 
Weitzman also evaluated opposing positions of several studies conducted regarding the 


importance of SAT scores and high school record relative to college academic 
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performance. Weitzman observed that there were several biases included in the chosen 
studies, but that overall the combined SAT may have a predictive validity of 0.62, an 
incremental validity 0.17 above and beyond that of high school record alone. This seems 
to lend credence to the argument that discrepancies in grade determination among 
different high schools introduces a certain amount of error when compared to the use of 
SATs relative to the college academic environment. Conclusions also indicate that the 
SAT is a valid instrument of selection for college applicants further strengthening the 
argument to utilize some form of SAT score in any model that attempts to predict college 
academic performance. 

Astin, (1968) supports the idea that the high school record is the single most 
effective predictor of academic performance, but aptitude tests, college selectivity and 
personality characteristics add significantly to the correlation with academic 
performance. 

Weinstein, (1987) observed that self-testing characteristics, or at least the idea 
that an individual performs some form of self-check prior to demonstrating required 
knowledge is a positive correlate for academic performance. Factors like discipline, 
focus and desire could also speak to academic performance in a similar manner. 

In summary, several studies have observed positive correlations between 
subjective and objective measures with academic performance. Since the LASSI focuses 
primarily upon subjective factors of performance, is currently implemented during the 
Naval Academy indoctrination process and archived in a database, it makes perfect sense 
to study this inventory as it pertains to academic performance along with all other 


objective criteria utilized by the Naval Academy Academic Center. Existing research 
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strongly suggests that objective criteria such as SAT and High School Record also be 


included in any model predicting academic performance. 


F. MODEL JUSTIFICATION 


In reviewing several of the studies summarized above, a general model for 
predicting academic achievement was developed. This model, presented below, included 


both subjective and objective variables in basic linear regression format. 


GPA = Bo+ Bi(SATM ) + Bx(SATV ) + B3(ClassStand ing) + 


(1) 
B4-13(LASSI Variables) +e 0 


In total, thirteen independent variables were included in the model designed to 
predict academic GPA, or CQPR. It was expected that not every variable would prove to 
be predictive of academic performance and that some would prove to be negative 
predictors. It was also expected that there would be a certain amount of interaction 
among some of the more closely related variables. However, this model is a logical 
starting point based upon the goals of this study and collection of previous studies 


reviewed. 


G. CHAPTER SUMMARY 


It is clear that there are numerous opinions as to what factors actually influence 


academic performance during the first year college. The United States Naval Academy 
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environment offers a unique opportunity to study a strictly regimented population where 
class attendance is mandatory and the amount of after school distractions during the first 
year is somewhat limited for underclassmen by a regimented study period. Couple this 
environment with very similar first-year course loads and you have a relatively good 
population for academic observation. The model proposed above will not, in all 
likelihood, provide the researcher with the ability to predict the CQPR for an individual, 
but it is hoped that those interested in identifying students who may experience academic 
difficulty may find a refined version of this model useful in assigning students to 


assistance programs. 


ZA 
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Il. DATA DESCRIPTION AND METHODOLOGY 

A. INTRODUCTION 

The purpose of this study is to empirically identify variables that significantly 
affect plebe year academic performance. By collecting historical case data representing 
individual midshipmen as they progressed through the Naval Academy in past years, this 
study attempts to quantify specific variables, both objective and subjective, that may 
affect academic performance. It is hoped that by quantitatively identifying these 
variables, a model can be developed that sufficiently predicts academic performance. As 
a result, the Naval Academy Academic Center could utilize this model to assist in the 


identification of midshipmen who may require academic assistance. 


B. DATA 

The data utilized in this study are primarily objective in nature. Initial entry data 
such as high school rank and size, high school GPA, SAT scores and other objective data 
were combined with subjective variables from the Learning and Study Strategies 
Inventory (LASSI) to form a main database catalogued by case using the midshipmen 
identification code, or alpha code. Descriptions of specific data characteristics are 
included below. 

The Institutional Research department of the United States Naval Academy 
provided a majority of case data obtained. Demographics, standard test scores, high 
school ranking information, academy academic performance as well as a variety of other 
variables are collected and maintained within this department’s databases. Data was 


collected on approximately 9800 midshipmen from the classes of 1996 through 2003. 
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This data was obtained in text format and transferred into the SPSS 9.0 statistical analysis 
program for manipulation and analysis. 

Another major source of information was the Naval Academy Academic Center. 
LASSI information was obtained in hardcopy and computer based formats for the classes 
of 1996, 1997 and 2001 through 2003. No data was available for the classes of 1998 
through 2000 due to technical difficulties realized upon retiring the Naval Academy 
Timesharing System that housed some of the LASSI data. LASSI computer based 
information was read directly from scan able test answer sheets, catalogued by alpha code 
and maintained on floppy disks for each class. The raw form of this data contained 
numerical values for each question answered resulting in a total of 77 fields of numerical 
data, one field for each question. This data was read into SPSS 9.0 for each class with 
variables created for each of the 77 questions on the test. Once common variables were 
established for each grouping of class data, the classes were merged to form a database of 
LASSI information. This database included over 6000 cases from the classes of 1996, 
1997 and 2001 through 2003. By itself, this form of information would not be valuable 
because the LASSI test requires that this raw information be manipulated according to 
instructions contained in the LASSI user’s manual. Only when these procedures are 
followed does the information become useful. Simply put, 77 questions were 
manipulated into 10 categories of the LASSI; attitude, motivation, time management, 
anxiety, concentration, information processing, selecting main ideas, study aids, self 
testing and test strategies. Following the included instructions, these categorical scores 
were computed and then converted into percentile format for comparison with nationally 
developed norms provided by the LASSI. This percentile form was then merged using 
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midshipmen alpha codes as a common variable between the original database and the 
LASSI database. Essentially, ten new variables based upon the subjective criteria of the 
LASSI were added to the objective data obtained from institutional research. 

Finally, alpha codes for all midshipmen having participated in the PIP were 
obtained, transferred into a spreadsheet format and then merged into SPSS 9.0 with the 
rest of the database. Plebes participating in the PIP were separated into three groups. 
The first group consisted of midshipmen who were originally selected for participation in 
the program prior to induction day. The second group, or add-on group one, consisted of 
those midshipmen selected for the program based upon their six-week grades. The third 
group, or add-on group two, consisted of those midshipmen selected for participation 
based upon their first semester QPR. It is important to distinguish between these three 
groups to assess the impact of timely intervention by the Academic Center. Once these 
variables were assembled, the database was considered complete and the analysis phase 
of study could proceed. Further discussion of individual variable format, description and 


use is included as part of the data analysis chapter. 


C. VARIABLES 


Several groups of data were collected and grouped into variables to assess their 
potential as predictors of academic performance. All of the data collected was combined 
into a master database catalogued into cases by Midshipmen alpha code. The following 
variables were assembled: ALPHA, HSRANK, HSSIZE, STANDING, SATV, SATM, 


WMMLT, ATTRITIO, SIQPR, S2QPR, CQPR, ATT, MOT, TMT, ANX, CON, INP, 
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SMI, STA, SFT, TST, RESTBRIG, INITIAL, ADDON], and ADDON2. Each of these 


variables is discussed in detail below. 


1. Independent Variables 

Independent, or explanatory variables are those variables that help predict a given 
dependent variable. It is critical to understand the process and format of each 
independent variable so that any interaction or lack thereof can be established with 
reliability. Basic descriptions of each candidate variable are summarized below. 

Variable ALPHA is the six digit string variable used to catalogue and link all 
other data sources to the master file. All data added to the master file was first 
catalogued by case using the midshipmen alpha code, organized in ascending order and 
then merged into the master file using ALPHA as the case identifier. This variable was 
used solely for cataloguing purposes and was not a candidate for prediction models. 

HSRANK and HSSIZE both relate to midshipmen high school academic 
performance and are numerically formatted. Neither variable is useful in itself due to the 
significant variations in relative class sizes. A number one ranking in a class of 10 is 
significantly different than a number one ranking in a class of 200. In order to make 
these variables meaningful, a third variable, STAND, was computed using equation 2. 

STAND = (HSRANK / HSSIZE) * 100 (2) 
This variable has relevance in that it determines relative position within a given high 
school class regardless of size. STAND is a numerical variable presented in percentage 


format. A lower value for this variable indicates higher academic performance relative to 
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classmates. This variable was considered as a candidate for prediction models due to its 
universal applicability and direct relationship to previous academic performance. 

SATV and SATM are both numerical variables representing the highest score 
earned on the Scholastic Aptitude Test prior to entering the Naval Academy. These 
variables are widely believed useful in determining potential academic success and are 
highly weighted by the Naval Academy Admissions Board. Therefore, the Naval 
Academy Admissions Office seeks applicants with scores of at least 1200 for the 
combined SAT. Since the Academy curriculum is highly math-based, the board also 
seeks an even split between math and verbal or a slightly higher math score. While there 
are no strict cut-offs for acceptance to the Naval Academy, a wealth of research 
summarized in literature review, points to the use of these two variables as predictors of 
academic success potential. Both variables were considered as candidates for the 
prediction model. 

WMMLT refers to the Admissions Office Whole Person Multiple. This number 
is in numerical format and is generated each year by the Admissions Board using a 
formula developed through an outside statistical analysis firm. The formula takes various 
objective entry criteria and gives each variable its own weight according to a historical 
database. There are several subjective factors that can affect this number such as teacher 
recommendations, military tradition within the family, and interviews results. This 
variable was not considered for use in developing prediction models due to continuous 
variations in the formula used to generate the WMMLT and the use of subjective factors 


that may vary from midshipman to midshipman. Since SAT’s, high school academic 
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performance, and many other entry variables are used to generate the WMMLT, this 
variable could also cause correlation problems among the other variables. 

ATTRITIO refers to the attrition code assigned to midshipmen leaving the 
academy. This code was originally received from institutional research consisting of a 
two-digit code for any midshipman leaving the Naval Academy for any reason. There 
are a multitude of reason for leaving the academy, but they can all be grouped into seven 
general categories: plebe summer voluntary resignation, academic year resignation, 
qualified resignation, academic discharge, discharge (conduct or performance), medical 
discharge, and deceased. This variable was not considered a candidate for prediction 
models because it deals solely with midshipmen attrition once it has happened and has no 
direct effect on academic performance. This variable was however useful as a filter for 
the descriptive analysis section. 

The next ten variables represent the ten results obtained from the LASSI test 
given during Plebe Summer. Each of these variables is numeric and significant to one 
decimal place. The variables are reported as percentiles representing midshipmen 
performance in each of the ten categories relative to historically tracked national norms. 
For ease of analysis, scores were compiled into ten equal percentile ranges between 10 
percent and 100 percent. A midshipman earning a score of 90 on any of the categories 
would have scored higher than 90 percent of those who had taken the test. Therefore, a 
higher score represents a higher level of potential in that specific area. All of these 
variables were considered for prediction models because they attempt to quantify 
subjective personality traits ignored by other objective criteria. A brief description of 
each LASSI variable is included below. 
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ATT, or LASSI attitude percentile measures an individual’s attitude toward 
school. “If the relationships between school and their life goals are not clear, then it is 
difficult to maintain a mind-set that promotes good work and its related tasks.” 
(Weinstein, 1987) Students scoring lower on this measure may have problems with 
scholastic identity. In other words, they may not understand why they are attending 
school or have a clear idea of how education fits into their lives. 

MOT assesses a student’s motivation to perform specific tasks related to 
achievement. Tasks such as completing assigned readings and homework, studying 
diligently, and accepting responsibility for performance relate directly to motivation and 
are key factors for this variable. If students learn to hold themselves responsible for their 
own study habits and performance, then they are less likely to assign performance 
outcomes to outside factors such as bad teachers, books or luck. This increased sense of 
responsibility and diligence would be accompanied by increased motivation toward 
school and a higher percentile score for MOT. 

TMT refers to student time management abilities. “Most students have various 
demands on their time and only by creating realistic schedules and sticking to them can 
they fit in everything.” (Weinstein, 1987) This statement holds especially true for plebe 
midshipmen who are under more stringent time constraints and pressures than the rest of 
the brigade. A midshipman scoring in the low range on this measure could be expected 
to fall short in academic or professional performance, experience a high degree of stress 
and earn a reputation for disorganization. 

ANX measures a student’s anxiety level toward academic tasks. Anxiety may be 


a determining factor in academic performance. “Cognitive worry, a major component of 
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anxiety, is manifested in negative self-referent statements.” (Weinstein, 1987) This 
negativity can be self-fulfilling and cause major problems with performance. If students 
believe they will perform badly on a test they may increase the likelihood of a negative 
outcome. Time consumed by anxiety is time taken from concentrated study efforts 
further complicating academic tasks. 

CON deals with a student’s level of concentration. People have different levels of 
focus, or concentration, whereby they are able to block outside interferences or internal 
thoughts and emotions in order to deal with a given situation or learning environment. A 
person who scores high in this area is able to concentrate on the subject without losing 
focus or allowing other interferences to wrrupt the learning environment. A person 
scoring in the lower percentiles may be easily distracted, emotional, and otherwise unable 
to maintain focus on a given subject. Obviously, a person scoring in the lower 
percentiles may have difficulty learning course materials if they cannot maintain an 
appropriate level of concentration. 

INP deals with how a person processes information. It deals with information 
organization and retrieval. This can take the form of both internal and external 
organization such as note taking and outlining, making lists, paraphrasing or creating 
internal pneumonics to remember important facts or figures. The degree of focus upon 
certain aspects of information can affect how much or how long that information is 
retained and this has specific implications with respect to academic achievement. For 
instance, if a student regularly “crams” for exams by memorizing large quantities of 
information in a short period before the exam, then the chances of truly learning the 


material are far more remote. This can be especially detrimental when students are 
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required to maintain a base level of knowledge for future courses. There are various 
ways to process information, but it is hoped that by comparing these differences against a 
historical database, that this variable may help predict academic performance. 

SMI refers to a student’s ability to identify main ideas from a given media source. 
This ability is particularly important to effective learning in a_ time-constrained 
environment such as the Naval Academy. “If a student cannot select out the critical 
information then the learning task becomes complicated by the huge amount of material 
the individual is trying to acquire.” (Weinstein, 1987) Identifying key study points and 
excluding redundant material promotes effective and efficient study and can be expected 
to positively affect academic performance. 

STA measures a student’s ability to effectively organize main ideas into useable 
study aids. It also measures ability to utilize existing materials to further enhance study. 
Students experiencing problems with timely and effective study habits may not be using 
tools such as study sessions, outlining, study guides, charts, diagrams and other such 
media to enhance their efforts. These students would be expected to earn low scores in 
the study aids variable. 

SFT is a self- testing and reviewing measure. This variable assesses a student’s 
ability to track their progress during the learning process through self-administered 
performance assessments. This ability to review personal retention and performance 
characteristics is useful in determining whether current study habits and strategies are 
sufficient for student goals or need to be revised for effectiveness. A student scoring 
highly for this characteristic uses mental reviews, problem solving, and question and 


answer techniques to consolidate new knowledge and perceive potential difficulties with 
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specific material or study strategies. This ability may have significant implications for 
academic performance. 

The last of the LASSI variables, TST, deals with test taking strategies. A student 
who takes the time to prepare not only for the material covered by a test but also for the 
type of test may score highly on this percentile. Methods of study, memory drills, and 
mental attitude toward tests all play a significant part in the test strategy variable. Test 
format can also be very important because effective study techniques can be very 
different based upon the format of the test. Memorization of small snippets of 
information may not be sufficient for the depth of knowledge required for an essay exam. 
Similarly, broad sweeping knowledge of a particular study area may not be necessary for 
a multiple-choice exam and may only serve to overwhelm the student. Test taking 
strategies may affect academic performance. 

INITIAL, ADDON1, ADDON2, and RESTBRIG are categorical variables 
created to identify variations in academic performance between midshipmen participating 
in the Plebe Intervention Program at three different time periods, and those not 
participating in an intervention program. It is expected that entering such intervention 
programs would be beneficial to those participating. However, comparing these 
midshipmen to the rest of the brigade provides a control variable by which the 
intervention midshipmen can be compared and the effectiveness of the intervention 
program can be simply analyzed. These variables are numerical and binary. For each 
case a value of “1” indicates that the midshipman was a member of one of the groups. A 
value of “O” indicates that the midshipman was not a member of that particular group. 
RESTBRIG carries a value of “1” for all midshipmen who had no intervention. INITIAL 
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carries a value of “1” for all midshipmen who were initially selected for participation in 
the intervention program prior to academic year. ADDONI] carries a value of “1” for all 
midshipmen who were added to the program after the first six-week grading period. 
Lastly, ADDON2 carries a value of “1” 6r all midshipmen added to the intervention 
group following the first semester grading period. For each case only one of these four 
variables could carry a value of “1” thereby establishing four distinct groups of 


midshipmen and allowing for comparisons among each group. 


2. Dependent Variable 

S1QPR, S2QPR and CQPR all refer to midshipman Quality Point Rating. This is 
effectively a midshipman’s academic grade point average. SIQPR, S2QPR, and CQPR 
reflect a midshipman’s I semester, 2" semester and cumulative grade point average 
during plebe year respectively. These variables are numerical in format and significant to 
two decimal places. S1QPR and S2QPR were useful for descriptive analysis and possibly 
for prediction models. CQPR was considered for prediction models as the dependent 
variable because it encompasses grades for both I and 2" semester grading periods. 
CQPR is numeric, significant to two decimal places and provides the most basic measure 
of academic performance consistent throughout the Naval Academy and the nation as a 
standard measure of performance. While there are subtle difference in grading criteria 
and calculation among educational institutions, the basic premise of study and 
performance assessment are similar so that comparisons may be accomplished and 


studies sited with little fear of conflict. If nothing else, CQPR provides a relatively stable 
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variable, within the walls of the Naval Academy, on which to base this statistical 


prediction model. 


D. GENERAL DATA OBSERVATIONS 

Over 9800 data cases were collected for possible use in developing academic 
prediction models. In order for a case to be useful, however, it would have to contain 
valid data for each variable included in statistical calculation. Any cases missing data 
entries for the included variables would automatically be excluded from the calculations. 
Therefore, it is very important to identify and attempt to explain any significant 
discrepancies or variations among the collected data. 

The most significant discrepancy noted in the collection of data for this study was 
that no LASSI information was available for midshipmen in the classes of 1998 through 
2000. Since these midshipmen represent approximately one third of all data collected 
and are located midway through the 1996 to 2003 population of useable data, it is 
important to address any data variations between the 96-97 group and the 01-03 group. 
This analysis is important because any inconsistencies between these groups can affect 
regression results and consequently affect the prediction model. _In order to address 
these issues, a comparison of mean values was constructed for all variables considered as 
prediction model candidates. Cases were separated into three case- groups based upon 
year so that mean value comparisons could be conducted. Each case-group was 
compared to the other case groups as well as all cases combined. The results of this 


comparison are presented in Table 1. 
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A significance level of 5.0% was chosen to represent areas of concern between 
case-group values. Class standing showed a significant increase of 7.4% between the 
1996-1997 and 2001-2003 groups although the maximum deviation between any of the 
case-groups and the overall mean was no greater than 4.1%. Too many variables exist 
among educational programs throughout the country to adequately assess this variation. 
One possible explanation for the increase could be more stringent high school class 
standing requirements at the academy admissions level. SAT Verbal scores also reflected 
an increase on the order of 12.4% between the 1996-1997 and 2001-2003 case-groups. 
This variation is easily explained by the renorming of the Scholastic Aptitude Test 
(SAT). This process, initiated several years ago renormalized the verbal portion of the 
SAT resulting in higher scores on the verbal portion only. The SAT verbal score mean 
values reported for case-group 1996-1997 reflect norms before this change was 
implemented. Mean values reported in case- group 1998-2000 reflect the interim period 
during the change and case-group 2001-2003 reflects mean values after the change was 
fully implemented. The fact that the SAT verbal values collected were based upon 
different test criteria should have minimal effect upon the overall prediction model. 
Studies have shown that SAT verbal is a much weaker indicator of academic 
performance than other variables such as SAT math or high school rank. (Collins, 1982) 

There was no significant variation in academic performance between the case- 
groups. As table 1 illustrates, there was only a 1.5% and 0.75% maximum differential 
between case-groups for the second semester and cumulative academic reporting periods 
respectively. The largest differential, 3.7%, occurred during the first academic reporting 


period. This larger differential, while still insignificant is somewhat expected due to the 


4] 


variability in a student’s initial adjustment to both the military and academic environment 
of the Naval Academy. 

Three of the ten LASSI variables showed significant variations between case- 
groups. Information Processing, Study Aids, and Self Testing categories varied 8%, 10% 
and 11% respectively between case-groups 1996-1997 and 2001-2003. Since no LASSI 
information was available for case- group 1998-2000, a trend analysis from 1996 to 2003 
could not be conducted. However, since there were no significant variations in CQPR 
throughout the period of study; the highest variation was 3.7%, it can be assumed that the 
degree of variation among these three variables over time had little affect upon overall 
CQPR and will have minimal, if any, negative effect on the accuracy of prediction 
models. The following text includes some general data observations. 

Midshipmen are required to maintain a 2.00 CQPR in order to successfully 
graduate from the Naval Academy. During the first semester of Plebe Year, 83.9% of all 
midshipmen studied earned a CQPR of 2.00 or above. 80.7% earned above a 2.00 for the 
second semester and by then end of Plebe Year, 86.1% of all midshipmen studied had 
earned at least a 2.00 cumulative CQPR. Of note is the fact that case- groups 1998-2000 
and 2001-2003 indicated a decrease in second semester performance relative to first 
semester performance. This is interesting because one would think that with the shock of 
both the military and academic environments facing new midshipmen, there would be a 
natural adjustment period during which any compromise in academic performance would 
appear. This should appear in the first semester grades if at all. However, this trend 
shows just the opposite effect except in case-group 1996-1997. In this case-group, 


second semester grades were higher than first semester grades. It is also important to 
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understand that this assessment is made using mean values, not individual values for each 
midshipmen and the variance between first and second semester grades is minimal. 
Another interesting point of observation is the fact that mean values for the 
LASSI variables fell high within normal ranges of 50 to 75. This means that on average, 
midshipmen scored higher than roughly 65% of the population in all categories with 
some variation depending upon the variable in question. In particular, midshipmen 
scored an overall average of 73.43% on the concentration category. Very simply stated, 
midshipmen generally score higher on the LASSI survey than the rest of the population. 
For any given LASSI category only 25% to 35% of midshipmen studied earned lower 


than average Scores. 
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Table 1 


Comparison of Variable Mean Values by Class Group 


Case Group 

Class Standing 
SAT Verbal 

SAT Math 

1 Semester Grades 
2nd Semester Grades 
Cumulative Grades 
Attitude 
Motivation 

Time Management 
Anxiety 


Concentration 


Information Processing 


Selecting Main Ideas 
Study Aids 
Self Testing 


Test Strategies 


All 


13.21 


599 


660 


2.67 


2.64 


2.67 


67 


66 


61 


67 


Ve) 


64 


65 


62 


65 


67 


1996-1997 


12.73 


566 


661 


2.61 


2.66 


2.66 


69 


66 


63 


68 


73 


67 


66 


66 


70 


67 


Note. a. Dashed entries represent missing data. 
b. LASSI scores are presented in percentile format 
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1998-2000 


13.02 


585 


656 


2.67 


2001-2003 


13.75 


636 


663 


Zid 


2.62 


2.68 


66 


66 


60 


67 


Tp 


62 


64 


59 


62 


67 


E. CROSS-TABULATIONS 


Cross-tabulation analysis allows the researcher to identify simple trends between 
both dependent and independent variables. In order to assess the viability of each 
independent variable discussed in previous sections, a cross-tabulation was conducted 
against cumulative plebe year GPA (CQPR). Variables, SATV, SATM, STAND, and all 
ten LASSI variables were crossed with CQPR for this preliminary analysis. This simple 
tool provides correlation statistics, graphical, and tabular data that can help determine 
whether the subject variable should or should not be used in prediction models. 

When cross-tabulation was conducted, some cases were excluded due to missing 
data. Valid cross-tabulations were obtained for 89% of both SATM and SATV scores, 
78% of STAND and approximately 50% of the LASSI variables. The lower percentage 
of valid LASSI cross-tabulations is due to the inclusion of cases from classes 1998 
through 2000 where no LASSI information was available. Two specific statistical values, 
the chi-square statistic and Pearson simple correlation coefficient (R), are generated from 
this analysis and can give the researcher some insights into the relationship between the 
crossed variables. The Pearson correlation coefficient is a measure of the strength and 
direction of the linear relationship between two variables. Its maximum absolute value is 
1.0. This means that variables that are perfectly correlated positively or negatively would 
have an R-value of 1.0 and —1.0 respectively. Any variations from these extreme values 


indicate degrees of correlation. 
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The chi-square statistic indicates the likelihood that the null hypothesis is true that 
any two given variables are independent of one another. It compares a calculated Chi- 
Square value against the static Chi-Square distribution in order to assess variance of the 
error term. A significance level must be chosen to determine what level of variance is 
acceptable. For the purposes of this study a significance level of 0.05 was used to assess 
whether or not to accept the null hypothesis. If the significance is greater than 0.05, then 
the null hypothesis is accepted. 

Cross-tabulation of SATV against CQPR yielded a chi-square value of 0.000 and 
an R-value of 0.286. A categorical review of the cross-tabulation indicated that the bulk 
of midshipmen (81%) earned an SATV score between 500 and 700 and that the higher a 
midshipman scored on the SATV, the more likely they were to earn a higher CQPR. This 
became evident when the observed number of midshipmen earning higher grades 
increased as SATV score increased. For example, the largest group of midshipmen 
(35%) scoring between 500 and 550 on the SATV earned a CQPR between 2.00 and 
2.50. The largest group of midshipmen (28%) scoring between 650 and 700 on the 
SATV earned a CQPR between 2.50 and 3.00. This indicates a possible relationship 
between SATV and CQPR. The chi-square statistic is less than 0.05 suggesting rejection 
of the null hypothesis that SATV and CQPR are independent of one another. An R-value 
of 0.286 indicates that while there may be a significant relationship, it is probably fairly 


weak. See Table #2 for Cross- Tabulation scores. 
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Table 2 


Cross-Tabulation of SATV Against CQPR 


SATV score category 


CQPR 550 600 650 700 

4.00 3.8 a3 9.9 15.3 

3:50 15.2 18.8 20.7 26.6 
3.00 29.6 *30.7 “31.6 OT pes 

2.50 235.1 29.3 25.4 22.6 

2.00 13.7 12.4 10.8 Ye 

1.50 2.4 1.5 L3 0:7 

1.00 0.2 0.0 02 0.0 

% of Cases 17.6 24.3 23.2 15.9 

°81.0 


Notes. Values represent percent of midshipmen within SATV 
Category who earned corresponding CQPR. 

81.0% of midshipmen earned SATV scores between 500 and 700 
“indicates the largest percentage of midshipmen within SATV category. 


"indicates total percentage of all cases 
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Cross-tabulation of SATM against CQPR yielded a chi-square value of 0.000 and 
an R-value of 0.419. A categorical review of the cross-tabulation indicated that the bulk 
of midshipmen (88.8%) earned an SATM score between 550 and 750 and that the higher 
a midshipman scored on the SATM, the more likely they were to earn a higher CQPR. 
This became evident when the observed number of midshipmen earning higher grades 
increased as SATM score increased. For example, the largest group of midshipmen 
(37%) scoring between 550 and 600 on the SATM earned a CQPR between 2.00 and 
2.50. The largest group of midshipmen (31%) scoring between 700 and 750 on the 
SATM earned a CQPR between 3.00 and 3.50. This clearly indicates a possible 
relationship between SATM and CQPR. The chi-square statistic is less than 0.05 
suggesting rejection of the null hypothesis that SATM and CQPR are independent of one 
another. The R-value of 0.402 indicates there may be a significant and fairly moderate 
relationship between the two variables. This observation concurs with expectations based 
upon previous studies discussed in the literature review. Refer to Table 3 for comparison 


results. 
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Table 3 


Cross-Tabulation of SATM Against CQPR 


SATM score category 


CQPR 600 650 700 750 

4.00 1.1 3.9 9.5 18.2 
3.50 7.6 14.6 23.8 *30.9 

3.00 2152, 30.2 sles 28.6 

2.50 *37.4 *35.0 26.2 17.1 

2.00 2351 14.6 7.9 4.4 

1.50 3.4 1.6 1.1 0.5 

1.00 0.1 0.1 0.1 O.2 

% of Cases 11.8 213 31.3 18.4 

°88.8 


Notes. Values represent percent of midshipmen within SATM category 
who earned corresponding CQPR. 

88.8% of midshipmen earned SATM scores between 550 and 750. 
“indicates the largest percent of midshipmen within each SATM category. 


indicates total percentage of all cases 
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A cross-tabulation was also conducted for STAND against CQPR. This resulted 
in a chi-square statistic of 0.000 and Pearson correlation coefficient of magnitude -0.376. 
Results of this analysis revealed that 77.8% of the midshipmen studied were in the top 
20% of their high school classes and over half were in the top 10%. Since STAND is 
organized so that a decrease in percentage value for high school class standing indicates 
an increase in high school academic performance, one would expect to see a 
corresponding increase in CQPR associated with a decrease in STAND. Therefore, a 
negative Pearson coefficient is expected since studies have shown that class standing in 
high school can be a predictor of college academic performance. (Astin, 1968; Lavin, 
1965; Weitzman, 1981) 

Analysis was focused upon the top 20% group since they represent the bulk of 
midshipmen and can be studied to identify major trends in performance. For midshipmen 
in this group, a decrease in STAND was met with a corresponding increase in academic 
performance. Specifically, 30% of midshipmen in the top 5% of their class earned a 
CQPR between 3.00 and 3.50 while 34% of midshipmen in the top 15% to 20% range 
earned a 2.00 to 2.50 CQPR indicating that there is indeed a relationship between the two 
variables. Since these four categories contain the bulk of midshipmen in the 1000 case 
random sample, there is a limited amount of information to be gained by analyzing 
statistics from the 25" percentile to the 100" percentile for academic standing. It should 
be noted however, that there does seem to be a downward trend in CQPR for the largest 
number of midshipmen in these lower categories that concurs with a negative association 
between STAND, in this format, and CQPR. For example, 41% of midshipmen in the 
25" percentile earned between a 2.00 and 2.50 CQPR while 41% of midshipmen in the 
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70" percentile earned between a 1.50 and 2.00. The negative Pearson Correlation 
coefficient concurs with these observations and indicates that there may be a valid and 
moderate association between the two variables in question. The chi-square statistic is 
less than 0.05 suggesting rejection of the null hypothesis that these two variables are 


independent of one another. Refer to Table 4 for comparison results. 
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Table 4 


Cross-Tabulation of STANDING Against CQPR 


STANDING (Percentile) 

CQPR 5 10 15 20 
4.00 19.7 5.8 4.9 3.6 
3.50 *29.9 213 17.0 14.1 
3.00 28.4 ss *33.8 28.2 
2.50 17.1 28.7 30.9 "335 
2.00 4.3 9.9 12.1 17.2 
1.50 0.6 0.7 1.4 3.0 
1.00 0.0 0.0 0.0 0.4 

% of Cases Sf 1959 13.1 | 
°77.8 


Notes. Values represent percent of midshipmen within STANDING 
category who earned corresponding CQPR. 

77.8% of midshipmen attained STANDING percentages in the top 20%. 
“indicates the largest percentage of midshipmen in STANDING category 


indicates total percentage of all cases 
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Cross-tabulations were also conducted for all ten of the LASSI variables in order 
to assess possible correlations with CQPR. In all but one category, Time Management, at 
least 50% of the midshipmen studied scored in the above average percentile range (70- 
100). For Time Management, 48.1% of midshipmen studied scored in the above average 
range. Conversely, anywhere from 24% to 40% of all midshipmen studied scored in the 
below average range for individual LASSI categories. Specific categories of note are the 
time management (TMT) and study aids (STA) categories of the survey at 40.1% and 
38.3% respectively. 

A significant portion of this general analysis is based upon observation of CQPR 
values for the largest grouping of subjects within each crossed variable percentile range. 
For example, if the largest number of midshipmen scoring in the 10%, 50%, and 100% 
ranges of the ATT category earn a 1.00, 2.50, and 4.00 CQPR respectively, then it is 
logical to assume that there is a significant relationship between these two variables given 
that the other percentile ranges reflect a similar relationship. Individual cross-tabular 
analysis is discussed in the following text. 

A cross-tabulation of LASSI attitude variable (ATT) against CQPR produced a 
chi-square statistic of 0.000 and Rvalue of 0.129. Results of the analysis indicate that 
59% of the subjects scored in the above average percentile while 32.4% scored in the 
below average percentile for this category. Observation of CQPR values for the largest 
subject group within each ATT category revealed no distinct patterns. However, the chi- 
square statistic of less than 0.05 suggests rejection of the null hypothesis that ATT and 
CQPR are independent of one another. An Rvalue of only 0.129 indicates that while 
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there may be a significant relationship between the two variables, it is probably very 
weak at best. 

A cross-tabulation of LASSI motivation variable (MOT) against CQPR produced 
a chi-square statistic of 0.000 and R-value of 0.256. Results of the analysis indicate that 
58% of the subjects scored in the above average percentile while 34% scored in the 
below average percentile for this category. Observation of CQPR values for the largest 
subject group within each MOT category revealed fluctuations between CQPR values of 
2.50 and 3.00 with a developing consistency toward 3.00 as motivation level increased. 
The chi-square statistic of less than 0.05 suggests rejection of the null hypothesis that 
MOT and CQPR are independent of one another. The Rvalue of 0.256 indicates that 
there may be a significant relationship between the two variables and it could 
significantly affect CQPR. 

Cross-tabulating the LASSI time management variable (TMT) against CQPR 
produced a chi-square statistic of 0.000 and an Rvalue of 0.109. Results indicate that 
48.1% of the subjects scored in the above average percentile while 40.1% scored in the 
below average percentile for this category. Observation of CQPR values for the largest 
subject group within each TMT category revealed a relatively constant value of 3.00 
across the board showing only minor fluctuations at the 20" and 30" percentile ranges. 
The chi-square statistic of less than 0.05 suggests rejection of the null hypothesis that 
TMT and CQPR are independent of one another. However, the low R-value of 0.109 
indicates that any relationship between the two variables has minimal if any significant 


effect on CQPR. 
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Cross-tabulating the LASSI anxiety variable (ANX) against CQPR produced a 
chi-square statistic of 0.000 and an R-value of 0.195. Results indicate that 57.3% of the 
subjects studied scored in the above average percentile while 29.7% scored in the below 
average percentile for this category. Observation of CQPR values for the largest subject 
group within each ANX category revealed a very slight upward trend in CQPR associated 
with increasing ANX scores. Specifically, a CQPR value of 2.50 was achieved for the 
largest single percentage of midshipmen scoring in the 10" through 40'" ANX percentile 
ranges. For the remaining ANX percentile ranges, the largest percentage of midshipmen 
earned a 3.00. The chi-square statistic of less than 0.05 suggests rejection of the null 
hypothesis that ANX and CQPR are independent of one another. However, the low R 
value of 0.195 indicates that any relationship between the two variables has only 
marginal effect on CQPR. 

Cross-tabulation of the LASSI concentration variable (CON) against CQPR 
produced a chi-square statistic of 0.000 and an Rvalue of 0.180. Results indicate that 
65.1% of the subjects studied scored in the above average percentile while 23.8% scored 
in the below average percentile for this category. There appears to be a fairly larger 
proportion of high scores for this variable as compared to other LASSI variables studied. 
Observation of CQPR values for the largest subject goup within each CON category 
revealed a very slight upward trend in CQPR associated with increasing CON scores. 
Specifically, CQPR values fluctuated from 2.50 to 3.00 between CON percentile ranges 
40 through 70 with a developing consistency toward 3.00 after 70%. This variable seems 
to somewhat mimic the performance of MOT with respect to CQPR. The chi-square 
Statistic of less than 0.05 suggests rejection of the null hypothesis that CON and CQPR 
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are independent of one another. However, the low R-value of 0.180 indicates that any 
relationship between the two variables has only marginal effect on CQPR. 

Cross-tabulating the LASSI information processing variable (INP) against CQPR 
produced a chi-square statistic of 0.044 and an Rvalue of 0.053. Results indicate that 
51.3% of the subjects studied scored in the above average percentile while 35.4% scored 
in the below average percentile for this category. Observation of CQPR values for the 
largest subject group within each INP category revealed no particular trends. CQPR 
values for these groups varied between 2.50 and 3.00 with no apparent pattern indicating 
little correlation. The chi-square statistic bordering significance at the 0.044 level 
suggests rejection of the null hypothesis that INP and CQPR are independent of one 
another. The low R-value of 0.053 indicates that any relationship between the two 
variables might be negligible. 

Cross-tabulating the LASSI selecting main ideas variable (SMI) against CQPR 
produced a chi-square statistic of 0.000 and an R-value of 0.129. Results indicate that 
54.2% of the subjects studied scored in the above average percentile while 33.2% scored 
in the below average percentile for this category. Observation of CQPR values for the 
largest subject group within each SMI category revealed a very slight upward trend in 
CQPR associated with increasing SMI value. The largest individual percentage group of 
midshipmen within the bottom half of the SMI categories generally earned a 2.50 CQPR 
while midshipmen in the top half generally earned a 3.00. Exceptions to this finding 
were observed at the 20'" and 90" percentiles where the largest individual percentage of 
midshipmen earned a 3.00 and 2.50 respectively. The chi-square statistic is less than 0.05 
suggesting rejection of the null hypothesis that SMI and CQPR are independent of one 
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another. The R-value of 0.129 indicates that while the two variables are not independent 
of one another, any relationship that may exist between the two variables is fairly weak. 

Cross-tabulation of the LASSI study aids variable (STA) against CQPR produced 
a chi-square statistic of 0.003 and an R-value of —0.065. Results indicate that 53.5% of 
the subjects studied scored in the above average percentile while 38.3% scored in the 
below average percentile for this category. Observation of CQPR values for the largest 
subject group within each STA percentile category revealed a very slight downward trend 
in CQPR associated with increasing STA value. This variable is interesting in the fact 
that it indicates a negative correlation with CQPR. Taken literally, a midshipman who 
scores highly on this measure seems to have a very slight academic disadvantage relative 
to those who have lower scores. There are several ways to interpret this finding. One 
interpretation is that there is a negative correlation between the two variables. Another 
could be that there is some other unmeasured characteristic associated with the STA 
variable that causes a negative interaction between the two. One other explanation could 
be the time the test is administered. Variations in resources and training prior to the 
Naval Academy experience could induce response bias and therefore interfere with the 
accuracy of the STA measure. There appears to be very little pattern to the fluctuations 
of CQPR for the largest single percentage of subjects across STA percentiles. 
Fluctuations do not occur until the 60% range and alternate between values of 2.50 and 
3.00 from the 60% through 100% ranges. The chi-square statistic is less than 0.05 
suggesting rejection of the null hypothesis that these variables are independent of one 
another. However, the very low R-value of -.065 indicates that any relationship may be 
negligible. 
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Cross-tabulation of the LASSI self- testing variable (SFT) against CQPR produced 
a chi-square statistic of 0.527 and an R-value of 0.038. Results indicate that 55.9% of the 
subjects studied scored in the above average percentile while 36.6% scored in the below 
average percentile for this category. Observation of CQPR values for the largest subject 
group within each SFT percentile category revealed no significant patterns with respect to 
CQPR. The chi-square value of 0.527 is greater than 0.05 suggesting that the null 
hypothesis be accepted that CQPR and SFT are ndependent of one another. The very 
low R-value of 0.38 indicates very little if any correlation and concurs with the chi- 
square observation. 

Cross-tabulation of the LASSI test strategies variable (TST) against CQPR 
produced a chi-square statistic of 0.000 and an R-value of 0.241. Results indicate that 
55.7% of the subjects studied scored in the above average percentile while 35.0% scored 
in the below average percentile for this category. Observation of CQPR values for the 
largest subject group within each STA percentile category revealed a slight increase in 
CQPR associated with higher performance on TST. Specifically, the largest single 
percentage of midshipmen within the 60" percentile range and lower generally received a 
2.50 CQPR. The only exception to this observation occurred in the 40" percentile range 
where the largest group of midshipmen earned a 3.00 CQPR. The largest single 
percentage of midshipmen within the 70" percentile and above generally earned a 3.00. 
This indicates a slight, but significant upward trend in CQPR. The chi-square statistic is 
less than 0.05 suggesting rejection of the null hypothesis that these variables are 
independent of one another. The R-value of 0.241 indicates that there may be a fairly 
moderate correlation between TST and CQPR. 
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F. BASIC ANALYSIS METHOD 


This section provides a brief explanation of the analysis methodology and 
structure implemented in order to obtain results for discussion. 

Once the data collection phase was completed as detailed above, preliminary 
analysis of existing Academic Center selection procedures was conducted using 
descriptive statistics, frequency analysis and cross-tabulations within the SPSS 9.0 
program. This helped to identify basic relationships between observed data and current 
selection methods. These simple relationships also played a part in developing a 
hypothesis for future assessment. Once this step was completed, a linear regression 
model was developed using both the information obtained from previous research and 
preliminary analysis observed in this study. After running the model through a linear 
regression in SPSS, results were discussed in terms of overall significance. Certain 
elements of the first model appeared to have no relevance in the prediction of academic 
performance, but could not be arbitrarily removed from the equation. Therefore, further 
analysis and research was conducted leading to the development of a second 
hypothesized model for academic performance. A description of why certain variables 
were ncluded or excluded was required in order to ensure this new model could be 
accepted as a valid modification of the original. Discussion of all variables to be used 
finalized this section with particular attention given to formatting the final equation for 
use in the PIP selection process. In addition methods of comparison similar to the first 
regression were utilized to provide a sound statistical basis for the final model. After 
model justification was completed, the model was run, results observed and discussed in 


detail. 
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Finally, a method for utilizing the hypothesized model to assist the Academic 
Center in identifying midshipmen who may experience academic difficulty was proposed 
with discussion and comparison to the existing selection criteria. This analysis method is 


detailed in the Comparison of Data section of this study. 
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IV. HYPOTHESIS 

A. INTRODUCTION 

The Academic Center utilizes several criteria in selecting midshipmen for 
participation in academic intervention programs. Each year a watch list of approximately 
200 names is generated based upon the following initial entry criteria: admissions data, 
final Naval Academy Prep School (NAPS) grades, admissions board recommendations, 
Naval Academy Athletic Association input, standardized test results, the Nelson Denny 
Reading test administered during Plebe Summer and to some extent, the LASSI survey 
results. From this watch list, midshipmen are selected based on perceived need. There is 
no empirical process in place that assists in the selection process. If an equation could be 
developed that could adequately predict academic performance, it would be a very useful 
tool for empirically ranking midshipmen academically. The problem with attempting this 
task is that there are typically many subjective factors involved with academic 
performance that can hamper the accuracy of a prediction model using only objective 
criteria. Lavin, 1965 studied the affect of several subjective factors such as study habits, 
attitude, interest, anxiety, adjustment and aggression in an attempt to understand the 
intricacies of such intellective factors and their affect upon academic performance. His 
results indicated that many of these factors did in fact have both positive and negative 
affects upon academic performance and should be included in any model attempting to 
predict GPA. Therefore, the attempt must be made to incorporate both subjective and 


objective criteria into any model that seeks to accurately predict academic performance. 
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B. MODEL EQUATION 


In order to assess the predictive power of specific variables against CQPR, an 
equation must be developed and analyzed. This equation should be reflective of the goals 
of the study and be well based from past research in order to be of significant value. 
Based upon literature review of academic achievement theory and past studies on 
subjective and objective performance measurement, the following prediction model for 
CQPR was developed. 


COPR = Bo+ BiSATM + B2SATV +B3STAND + B4ATT + BsMOT + B EMT + 
B7ANX +B sCON + BoINP + BiSMI + BuSTA+ BeSFT + B uTST+€ 0 


(3) 

Next a hypothesis was required in order to establish a basis for review of findings. 
The hypothesis details expected signs, either positive or negative, for each independent 
variable coefficient. This allows the researcher to express an educated opinion as to what 
the observed results may reveal. This opinion is then either disputed or reinforced by 
observed regression results and either supports or negates the null hypothesis for each 
variable coefficient. Utilizing the information obtained from both the literature review 
and data description, a hypothesis was developed for each of the variables included in the 
prediction model. See equation 4. This hypothesis will be compared to the observed 
regression results for further discussion in the results chapter. 

+ + - + + + + + 7? + - + 4+ 


CQPR = f(SATM,SATV,STAND, ATT, MOT, TMT, ANX,CON, INP, SMISTA,SFT, TST) (4) 
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Three included variables, INP, STA, and SFT are of particular interest. Cross- 
tabulation data indicated that INP bordered upon significance at the 0.044 level with 
regard to the chi-square statistic and possessed a very low Pearson (R) correlation 
coefficient. Since there was relatively little valid information regarding the subject of 
information processing in both the data description and literature review areas of study, 
no assumption was made as to the sign of this regression coefficient although it was 
assumed that this variable would have minimal effect upon CQPR. Data obtained from 
crossing the STA variable with CQPR revealed a negative correlation coefficient 
although the Rvalue was very small and the chi-square statistic was not particularly 
strong. Most probably, this variable had no real predictive value for CQPR, but was 
included in the preliminary analysis to be reviewed following regression observations. 
Literature review indicates that the SFT variable is a positive correlate for academic 
performance. (Weinstein, 1987) However, cross-tabular analysis against CQPR revealed 
no such correlation. The chi-square statistic was insignificant and the R-value was very 
small. This variable is expected to carry little or no weight in the prediction model. 
SATM, SATV, and STAND were all expected to be positive indicators of academic 
performance based upon both the literature review findings and data description 
including cross-tabular analysis of each against CQPR. However, since STAND is 
formatted so that a lower value indicates higher academic performance the regression 
coefficient associated with this variable is expected to be negative. Both MOT and TST 
were expected to be fairly significant and positive variables in predicting academic 
performance due to their R values each approaching 0.300 and significant chi-square 


observations. Literature review indicates that there was inconsistency in research 
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findings on motivation as a predictor of academic performance, but that there was some 
evidence leading to the belief that a small correlation exists between motivation and 
academic performance. (Lavin, 1965) The other variables including ATT, TMT, ANX, 
CON, and SMI produced significant chi-square statistics with low Rvalues indicating 
that their involvement in the prediction of academic performance may serve to refine and 


not define the model. 


C. CHAPTER SUMMARY 


The model equation developed involves thirteen different variables. There is little 
doubt that attempting to utilize so many variables for prediction of such a dynamic 
dependent variable as cumulative grade point average may lead to such complicating 
factors as probable high correlations among some of the independent variables. This 
multicollinearity problem can affect the significance of coefficients of the highly 
correlated variables. 

It is important to keep in mind that a key point of this study is to help simplify and 
provide consistency throughout the selection process for students who may have 
academic difficulties. Even if the resulting equations developed through regression do 
not provide an effective means for predicting actual CQPR, they are still very useful if 
they can be utilized to provide more accurate and efficient identification of those who 


will otherwise experience difficulty. 


64 


V. RESULTS 

A. INTRODUCTION 

Implementing the model linear regression equation developed in the hypothesis 
section, linear regression analysis was conducted using the SPSS 9.0 software. In 
reviewing the results, it was important to establish significance level thresholds with 
respect to the observed t characteristic and their respective significance values. For the 
purposes of this study the following criteria was used to determine the level of 
significance for a particular variable. First, the calculated (t)) value is observed from 
regression results provided by SPSS 9.0. This t value is then compared against the 
critical t value (t,) in order to determine whether or not to accept the null hypothesis. If t 
has an absolute value larger than t, then the null hypothesis is rejected. In the hypothesis 
section of this study, hypothetical signs for each regression coefficient were developed 
based upon research data. Therefore, the null hypothesis for a particular variable 
regression coefficient would indicate a value of the opposite sign as the hypothesis for 
that coefficient or zero. The t, value to be used depends upon the level of significance 
chosen. For this study, the following scale was used to determine variable coefficient 


significance. Table 5 details coefficient significance figures. 
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Table 5 


Range of Significance for Regression Coefficients 
Range of Significance Prediction Value 
0.000 - 0.049 Highly Significant 
0.050 - 0.099 Significant 
0.100 - 0.149 Marginally Significant 
0.150 and higher Not Significant 
Note. Values are determined based upon 


infinite degrees of freedom. 


B. REGRESSION ANALYS IS 


Once the initial regression was completed using all ten LASSI variables, SATV, 
SATM and STAND, the results were compiled and analyzed for statistical significance 


using significance values in Table 5 as the standard. The following results were obtained. 


1. 1° Regression 

Of the entire database of information compiled consisting of nearly 10000 cases 
of data, 3998 held valid entries for all the variables included in this initial regression 
analysis. This is partially due to system missing entries for midshipmen with incomplete 
data, but mostly due to the missing LASSI data from the classes of 1998 through 2000. 
Therefore, this regression analysis represents the prediction of academic performance 


primarily for classes 1996, 1997, and 2001 through 2003. 
66 


Two of the statistical results obtained from regression analysis are the coefficient 
of determination (R’) and the F statistic. R’ is a goodness of fit statistic that helps 
describe how well the model equation fits the sample data. In other words, a value of 
1.00 for R? means that all of the error term is explained by residuals in the equation. This 
value is always positive and ranges between zero and one. A value of one indicates that 
the equation perfectly fits the sample data. Conversely, a value of zero indicates that the 
equation fits the data poorly if at all. The F statistic is a measure of the overall 
significance of an equation. Unlike R° that determines degree of fit, the F statistic 
provides a formal hypothesis test of the overall fit and can be a powerful indicator of the 
predictive power of an equation. 

For this initial model, the R® and F statistic observed were 0.324 and 146.59 
respectively. At first glance, the R seems low. However, this may indicate that the 
model equation may not predict CQPR very well for an individual, but may predict fairly 
well for the group as a whole. The F statistic observed was also significant indicating 
that the model does in fact possess a degree of predictive power for CQPR. Each 
independent variable’s performance is described next. 

Of the thirteen variables included in the linear regression model above, ten of 
them reflected some degree of significance with respect to CQPR. Specifically, the 
regression coefficients for STAND, SATV, SATM, MOT, and STA were all significant 
at the 0.000 level. Coefficients for ATT, TMT, SMI, and TST were all “highly 
significant” and varied from 0.003 to 0.019 in significance. Only one variable coefficient 
was evaluated as “significant” due to a t value of 1.693 and significance level of 0.091. 
While still significant at the 0.10 level, this coefficient lay on the border of marginal 
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significance and warranted further discussion and analysis before any decision was made 
on whether or not to remove it from future equations. ANX, CON, and INP, however, 
reported t values of 0.331, 0.456, and 0.636 respectively. These variable coefficients 
were considered “not significant’ because of their low t values and associated 
significance levels. These variables were only significant at the 0.74, 0.65 and 0.52 
levels respectively and lie well outside the parameters for even marginal significance. 
This indicates it is most likely that these variables do not assist in the prediction of 
CQPR. 

Some interesting results were encountered with respect to hypothetical coefficient 
signs. First, ATT was hypothesized to be a positive indicator of CQPR. However, 
regression results argue that this variable is a negative indicator of CQPR. At first 
glance, this result does not seem logical. One would magine that a student’s attitude 
toward school might have significant influence upon scholastic achievement. The 
attitude category measures clarity of educational goals and the importance of education to 
an individual. At the Naval Academy, there are several competing priorities that may 
affect the prediction polarity of ATT. The end result of graduation from the Naval 
Academy is an officer commission in the Navy or Marine Corps. A Midshipman’s Order 
of Merit (OOM), comparable to grade point average (GPA) in civilian institutions, is 
determined by several factors including academics, military and physical performance 
and conduct. This combination of competitive factors is very different from civilian 
institutions where academic grade point average alone is the primary measure of 
performance. At a civilian institution, a person who does not value education and lacks 


sufficient clarity of educational goals could be considered self-destructive. At the Naval 
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Academy, such a person may simply place higher value on another source of competitive 
measure. Midshipmen can often be heard stating that their performance as officers will 
not be determined by the value of their CQPR. That is not to say that midshipmen do not 
value education, only that there are other factors that compete with academics for 
priority. 

Another area of surprise was the SMI variable. It was hypothesized that the 
ability to select main ideas and know what areas of text to study would be a valuable 
asset and naturally affect CQPR in a positive manner. 

The opposite effect was observed in the regression equation. The coefficient of 
this variable was significant with a t value of —2.77; SMI was shown to be a negative 
predictor of CQPR. There could be several explanations for this result. Selecting Main 
Ideas may truly not influence academic performance or even be a detriment due to the 
focus on only certain material and not on the whole understanding. Another possibility 
could be that the SMI variable is highly correlated with another variable in the equation. 
This situation could cause interference in the relationship between SMI and _ the 
dependent variable. 

After analyzing results from the first regression it was determined that a second 
regression would be necessary to determine the overall model effect resulting from 
removal of variables found to be insignificant. Due to reasons summarized above, 
variables ANX, CON and INP were removed from the model. Although initially 
insignificant, variable SFT was retained in the equation in order because it bordered on 


significance. The effects of removing the other three variables could positively or 
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negatively affect the significance level of the SFI variable. Table 6 summarizes 


observed results from the first regression. 
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Table 6 


of Ist Regression Analysis for Variables Predicting CQPR 


Summ. 

(N = 3998) 

Variable B 
Constant -0.05333 
STAND -0.00901 
SATV 0.00126 
SATM 0.00285 
ATT -0.00114 
MOT 0.00399 
TMT 0.00107 
ANX 0.00014 
CON 0.00024 
INP 0.00023 
SMI -0.00120 
STA -0.00220 
SFT 0.00072 
TST 0.00112 


Note. 


R’ = 0.324, F = 146.59 


SEB 


0.109 


0.001 


0.000 


0.000 


0.000 


0.000 


0.000 


0.000 


0.001 


0.000 


0.000 


0.000 


0.000 


0.000 


a 


2. 2"4 Regression 

A second regression was performed in exactly the same manner as the first 
regression. Variables hypothesized as predictors of CQPR were once again assembled 
with three qualified exceptions as discussed previously. The primary difference between 
these two regression procedures was the removal of variables ANX, CON and INP based 
upon variable discussion from the first regression results. All other procedures remained 
constant. SPSS 9.0 was used to perform the regression of the dependent variable against 
the independent variables with the following results. The R’ value obtained from this 
regression was 3.27 with an F statistic value of 196.73. The removal of three 
insignificant variables did not affect the R’ squared statistic appreciably although a 
positive increase of 0.03 was observed. This indicates that the variables removed did not 
significantly affect the predictive power of the model equation. However, the change in 
the F statistic was more pronounced at an increase of 50.15. This means that although the 
predictive power of the equation did not change appreciably, the significance of the 
values obtained increased by 34%. Therefore, the removal of ANX, CON and INP 
resulted in a more statistically significant outcome. Table 7 summarizes results from the 


second regression. More in-depth discussion of regression results follows Table 7. 
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Table 7 


Summary of Linear Regression Analysis for Variables Predicting CQPR 


Second Regression: ANX, INP and CON Variables Removed. 


(N = 3998) 

Variable B SEB B t Sig. 
Constant -0.06114 0.107 - -0.57 0.568 
STAND -0.00898 0.001 -0.211 -14.89 0.000 


SATV 0.00127 0.000 OAS 10.69 0.000 


SATM 0.00286 0.000 0.295 19.84 0.000 


ATT -0.00099 0.000 -0.043 -2.64 0.008 
MOT 0.00392 0.000 0.166 8.61 0.000 
TMT 0.00113 0.000 0.049 219 0.005 
SMI -0.00111 0.000 -0.048 -2.64 0.008 
STA -0.00223 0.000 -0.105 -6.61 0.000 
SFT 0.00088 0.000 0.039 2.20 0.028 
TST 0.00123 0.000 0.054 2.86 0.004 


Note. R2 = 0.327, F= 196.73 
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Additionally, there were appreciable changes in the significance values of several 
included variables. Specifically, while all of the variables maintained the same 
coefficient sign as in the previous regression, there were notable differences in the t and 
significance values. The following text and table summarize changes. 

The t values for STAND, SATV, SATM, MOT, and STA varied by —0.17, 0.25, 
0.59, 0.11, and —0.32 respectively indicating a strengthening of the relationship between 
these independent variables to the dependent variable. These variables were all highly 
significant in the first regression and no appreciable changes in significance were 
observed for the second regression. ATT and SMI reflected a slight weakening in t 
value. ATT showed a 0.31 positive change weakening the negative t value from —2.95 to 
—2.64. This also changed the significance level from 0.003 to 0.008. Although this is a 
noticeable change, the variable remains highly significant relative to the dependent 
variable. Similarly, SMI showed a 0.13 positive change weakening the negative t value 
from —2.77 to —2.64. This changed the significance level noticeably from 0.006 to 0.008. 
However, the variable remains highly significant relative to the dependent variable. Of 
particular interest were the changes observed in the significance values associated with 
variables TMT, SFT, and TST. In each case, the results observed in the second 
regression reflected increasing t values corresponding to higher significance relative to 
the dependent variable. TMT and TST reflected moderate t value increases of 0.38 and 
0.52 respectively. Both of these variables were highly significant in the first regression. 
SFT provided the most interesting results between the first and second regression. As 
noted in the chart below, SFT bordered significance at the 0.09 level in the first 
regression with a t value of 1.69. Results from the second regression reflected a dramatic 
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change in t value from 1.69 to 2.20. This resulted in a change in characterization for 
variable SFT from marginally significant to highly significant at the 0.03 level. Possible 
explanations for this phenomenon are detailed below. 

The removal of ANX, CON, and INP for the second regression served to further 
strengthen the significance values of TMT, SFT, and TST. One possible explanation of 
this result is that removing some of the interactive complications associated with three 
norsignificant variables in the same equation allows the highly significant variables to 
more fully describe observed results. Correlation results from the first regression seem to 
support this theory. TMT was observed to be highly correlated to CON with a Pearson 
Correlation of 0.62. SFT was observed to be moderately correlated with both CON and 
INP with Pearson Correlation values of 0.41 and 0.49 respectively. TST was observed to 
be highly correlated with both ANX and CON with Pearson Correlation values of 0.59 
and 0.57 respectively. Because these variables were highly insignificant relative to the 
dependent variable and highly correlated to several highly significant variables, their 
removal could allow for a better overall description of the observed results. Table 8 


refers. 
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Table 8 


Summary of Changes in t Values and Significance 


Observed 


Between First and Second Regression Analysis. 


3998) 
Variable 


Constant 
STAND 
SATV 
SATM 
ATT 
MOT 
TMT 
SMI 
STA 
SFT 


TST 


Regression | 


t value 


-0.49 


-14.72 


10.44 


19.25 


-2.95 


8.50 


2.41 


-2.77 


-6.29 


1.69 


2.34 


Sig 
0.624 
0.000 
0.000 
0.000 
0.003 
0.000 
0.016 
0.006 
0.000 
0.091 


0.019 


Regression 2 


t value 


-0.57 


-14.89 


10.69 


19.84 


-2.64 


8.61 


219 


-2.64 


-6.61 


220 


2.86 
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Sig. 
0.568 
0.000 
0.000 
0.000 
0.008 
0.000 
0.005 
0.008 
0.000 
0.028 


0.004 


N= 


Change 

At DSig 
-0.08  -0.056 
-0.17. 0.000 
0:25 0.000 
0.59 0.000 
0.31 0.005 
0.11 0.000 
0.38 -0.011 
0.13 0.002 
-0.32 0.000 
0.51 -0.063 
0.52 -0.015 


Table 8 details changes in significance for all included variables in the CQPRpred 
model. Simply stated, Class standing and individualized SAT scores proved to be the 
most significant variables for predicting relative performance. This is not a surprise since 
literature review strongly supports these observations. Indicating tvalues of -14.72, 
10.44, and 19.25 for Class Standing, SAT Verbal and SAT Math respectively; these 
variables are, by far, the strongest predictors observed. It should also be noted that SAT 
Math was the most powerful predictor of all independent variables tested. Since Class 
Standing reflects overall high school performance in all academic fields and SAT scores 
are individualized to reflect Verbal and Math based performance, the relatively strong 
emphasis on technical, math, and science based courses at the Naval Academy supports a 
more significant correlation between SAT Math and academic performance. 

Even though SAT Math, SAT Verbal and Class Standing were observed as highly 
significant variables, several LASSI variables also proved to be highly significant. This 
is interesting because the LASSI is based upon personality characteristics and SATs and 
Class Standing are objective and performance-based. Backed by literature review and 
observations from this study, the inference can be made that, in conjunction with 
objective measures, subjective variables such as personality characteristics can provide 
moderately enhanced prediction capability to relative performance models of academic 
achievement. Specifically, the following LASSI variables were observed to be highly 
significant at the 0.00 to 0.05 level and are listed in order of precedence from most 
significant to least significant: Motivation, Study Aids, Attitude, Selecting Main Ideas, 
Time Management, and Test Strategies. Only Self Testing was observed as significant at 


the 0.10 level and was considered a moderate predictor for the model equation. 


Td. 


Three LASSI variables, Study Aids, Selecting Main Ideas and Attitude were 
observed to have a negative impact upon Cumulative Quality Point Rating. While there 
is no certain answer why above average performance in these areas would be correlated 
with a downward trend in academic performance, some thoughts are detailed in the 
following text. 

The Study Aids and Selecting Main Ideas variables are centered on the idea of 
outlining and compartmentalizing study in order to achieve a requisite knowledge level. 
Academic and military studies at the Naval Academy are more detail oriented and require 
a thorough understanding of material. Students are taught that attention to details is 
paramount to their success as Navy and Marine Corps Officers. Therefore, the ability or 
tendency to compartmentalize or outline material may be a detriment to a student in this 
environment. 

As discussed in previous chapters, the negative coefficient for Attitude could be 
explained by the dual focus of the Naval Academy. The purpose of the Academy is to 
provide Navy and Marine Corps Officers to the fleet. This is accomplished through a 
variety of academic and military classes, practical experience, training and immersion in 
a complete military environment while attending the Naval Academy. It is not 
uncommon for a midshipman to look at professional development and military courses as 
more important than purely academic pursuits. This is somewhat perpetuated by the fact 
that course of study has little to no bearing on most service options for midshipmen. In 
other words, a student is not required to study aerospace engineering in order to select 


Naval Aviation as their service community after graduation. Therefore, a negative 
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coefficient for Attitude does not mean that midshipmen have a negative perception of 
academics but may value other aspects of Academy life as more important. 

It is important to understand that personality characteristics are very difficult to 
quantify. The LASSI provides a statistically sound basis for understanding and 
evaluating personality characteristics, but there are always exceptions to the rule. With 
continuing efforts to maintain long term, consistent database information and conduct 
further trend analysis, the intricacies and relationships between each of these variables 


may be more fully understood. 


C. COMPARISON OF DATA 


After analyzing results from the second regression, a model was developed for the 
purpose of comparing the current PIP selection process to a process that includes an 
empirical equation derived from linear regression. This study focuses upon the 
prediction of CQPR. Therefore, any model derived from regression results should 
ultimately provide a value for CQPRprep (predicted CQPR) for comparison with 
population data. Since results from regression number two appeared to more fully 
describe population data as described in previous sections, the variables and associated 
coefficients from the second regression were utilized to develop the model below for 


CQPRprep. 


CQPRerep= (—6.114E — 02) +(—8.979E —03)(STAND + (1.268E —03)(SATYV) + 
(2.862E —03)(SATM) + (-9.902E — 04)(ATT) + (3.918E —03)(MOT) + 
(1.132E—03)(TMT) + (—1.108E —03)(SMD + (—2.227E—03)(STA)+ 
(8.779E —04)(SFT) + (1.229E —03)(TST) 
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Once a valid model was established, a process for comparison was devised in 
order to assess the feasibility of including empirical analysis in the Plebe Intervention 
Program selection process. Several aspects of the selection process required 
consideration for this study to have any significant value to the Naval Academy Academy 
Academic Center. Several questions had to be answered. What are the limitations of the 
empirical formula developed? Can the formula be implemented as stand-alone selection 
criteria, or should other factors play a role in the selection process? Does the empirical 
formula provide a more accurate assessment of academic performance than the current 
selection process? Answers to these questions are critical to determining whether to 
utilize this model, in part or as a whole, for practical use in the Academic Center’s 
selection process. 

As previously covered, the Naval Academy Academic Center places 
approximately 200 midshipmen on an academic watch list. This list is generated from a 
variety of sources including, but not limited to, SAT scores, reading comprehension tests, 
NAAA recommendations and other sources. Typically these individuals are ranked in 
order of concern although there is currently no empirical process in place to assist in their 
identification. During the period of study, the Academic Center had the capacity to 
accept approximately 100 midshipmen into the Plebe Intervention Program. This 
constitutes approximately half of those placed on the watch list. For the purposes of this 
study, it was critical to determine whether the incorporation of an empirical process 
provided an enhanced ability to select midshipmen who experienced significant academic 


difficulty. In order to determine the validity of the empirical process, the prediction 
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model was used to develop a predicted first semester Quality Point Rating or CQPRprep 
for each individual included in the study. 

To calculate a valid CQPRpregp, the collected data had to be filtered for invalid 
entries. This was accomplished by using the SPSS 9.0 filter function to remove all cases 
with invalid entries for the included variables and actual CQPR. After filtering, a total of 
4065 valid cases were assembled for comparison analysis. Using the model for 
CQPRprep developed from results obtained in the second regression, predicted Quality 
Point Ratings were calculated for each case. The next step was to determine how to 
compare the predicted CQPR to both actual CQPR and the PIP selection process. 

The object of both the Plebe Intervention Program and this study is to ensure that 
midshipmen most in need of academic assistance are identified. The measure of 
academic success utilized for this purpose is CQPR. Therefore, in order to satisfy both of 
these objectives, actual CQPR was used as a reference point. Those midshipmen with the 
weakest actual CQPR were identified as individuals most in need of academic assistance. 
Since the Naval Academy Academic Center possesses limited resources and provides 
services to approximately 100 midshipmen via the Plebe Intervention Program each year, 
it is logical that they would desire to intervene with those midshipmen projected to earn 
the weakest actual CQPR. Each year approximately 1100 midshipmen are inducted into 
the Naval Academy. If approximately 100, or 9.10 percent of these midshipmen 
participate in the Plebe Intervention Program at some point in time, then this percentage 
is valuable as a comparison point. 

Next, the variable ACINTERV was created for all included cases to represent 


participation in the Plebe Intervention Program. This variable is categorical indicating a 
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value of one (1) if the midshipman participated in the PIP and zero (O) if not. Once 
variables ACINTERV and CQPRprep were created, the final comparison was conducted. 
Assuming that 9.10 percent of the total cases participated in the PIP, a simple 
comparison was performed using the lowest 9.10 percent of values for both CQPR and 
CQPRprep. This was completed utilizing the SORT CASES function of SPSS to sort 
each variable in ascending order and filter out all but the lowest 9.10 percent of each. 
The result was a collection of 370 cases with the lowest predicted or actual CQPR 


respectively. Descriptives for CQPR and CQPRprep are included in Table 9. 
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experiencing academic difficulty. 


Table 9 


Comparative Analysis of COPR, CQPRpred, and 


PIP Selection Process (N=370) 


Descriptive Statistics within Actual CQPR 


Variable Valid(N) Minimum Maximum Mean 
CQPR 370 0.50 1.88 1.64 
CQPRpred 370 1.28 522 235 


Note. Table represents lowest 9.10 percent (370) within 


Actual CQPR. 


Isolating the lowest 9.10 percent of all observed values for CQPR and CQPRprep, 


Intervention Program and the accuracy of the prediction model developed. 


Ideally individuals included in the assistance program reflect individuals actually 
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Deviation 


0.22 


0.34 


provides a basis for simultaneously comparing the individuals included in the Plebe 


Of the 370 cases identified, 154 participated in the 
Plebe Intervention Program. This reflects a positive identification rate of approximately 
41.6 percent. By comparison, 121 cases included in the lowest CQPR group were also 
included in the lowest CQPRprep group indicating a positive identification rate of 


approximately 32.7 percent. This simple comparison indicates that the current selection 


process provides more accurate identification of individuals who experience academic 
difficulty. Specifically, the current process identifies those who have academic difficulty 
approximately 8.90 percent more effectively than the CQPR prediction model. Table 10 
details frequency of identification for academic assistance based upon the current 


Academic Center selection process and use of the CQPR prediction model. 


Table 10 
Comparative Analysis of CQPR, CQPRpred, and 
PIP Selection Process (N=370) 


Frequencies of Subject Variables Within CQPR 


Variable Valid (N) Frequency Percent 
ACINTERV 370 154 41.6 
CQPRpred 370 121 32] 


Note. Table represents lowest 9.10 percent (370) within Actual CQPR. 
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Does this mean there is no value added by utilizing the model equation? More in- 
depth analysis was able to shed some light on this question. 

It is important to realize that the positive identification rate of 41.6 percent 
reflects the total number of participants in the Plebe Intervention Program for the entire 
year. In other words, many of the individuals who participated in the program were 
identified after the first six-week grading period. Since previous research indicates 
earlier identification and intervention can provide more beneficial results toward 
academic success, it is of value to identify and utilize a selection process that focuses 
upon accuracy and timeliness. In order to assess the timeliness of the current selection 
process, the ACINTERV variable was then broken down further to indicate whether an 
individual was identified prior to or after the first six-week grading period. When 
developing the ACINTERV variable, several records of data obtained from the academic 
center archives were utilized. These records indicated when a midshipman entered the 
PIP. Midshipmen entered the PIP in one of three groups during the semester, the initial 
group, the first add-on group or the second add-on group. The initial group was 
identified prior to the start of the academic year. The first add-on group was identified 
after the first six-week grading period and the second add-on group was identified after 
the second six-week grading period. All three groups were developed into categorical 
variables INITIAL, ADDON1 and ADDON2 and combined to develop the ACINTERV 
variable. For the purpose of further comparison to the prediction model, a descriptive 
analysis was conducted using SPSS 9.0 to identify how frequently midshipmen were 
identified to participate in the PIP prior to starting academic classes. The same group of 


370 midshipmen with the lowest CQPRs was utilized for this comparison. 
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A total of only 69 midshipmen, or 18.6 percent within the CQPR group were 
identified prior to the start of academic year, while 32.7 percent were identified within 
the CQPRprep group. While this low identification rate is partially due to the Academic 
Center’s policy of using the first six-week grading period as a secondary feedback 
identification process, there is a significant amount of intervention time lost due to the lag 
in identification. Since the prediction model more accurately identifies midshipmen with 
academic difficulty, (by 14.1 percent) there is significant value-added to the intervention 
program in utilizing this model as a subjective supplement to the objective criteria 
currently in use. This would allow for a higher percentage of intervention early in the 
academic year, thereby providing the individual student more time to implement skills 
learned during intervention to positively affect CQPR. Table 11 details the frequency of 
identification for academic assistance based upon both the prediction model and the 
current Academic Center selection process including only those midshipmen who 
participated in the program during the first six-week grading period. It is important to 
recognize that there is a six-week delay in follow-on identification with the current 
Academic Center selection process and no delay in utilization of the CQPR prediction 
model. The current process requires this delay in order to obtain six-week academic 
performance feedback prior to selecting additional PIP participants. Utilizing the CQPR 
prediction model may alleviate some of this need and allow for most intervention 


participants to be identified prior to commencement of academic classes. 
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Table 11 
Comparative Analysis of CQPR, CQPRpred, and 


PIP Selection Process (N=370) 


Frequencies of Subject Variables Within CQPR 


Variable Valid (N) Frequency Percent 
INITIAL 370 69 18.6 
CQPRpred 370 121 32.7 


Note. Table represents lowest 9.10 percent (370) within Actual CQPR. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 
A. CONCLUSIONS 


It is abundantly clear that exclusive use of the prediction model developed within 
this study for identification of all students who participate in the PIP is inappropriate. 
There are many subjective elements affecting CQPR that cannot be fully addressed by a 
simple equation. It is also clear that this equation cannot be used to accurately predict 
actual CQPR for an individual with any significant degree of accuracy. This is evidenced 
by the low R' square value of 0.327. However, there is sufficient evidence to support 
implementing the CQPR model developed above for the purpose of establishing potential 
relative performance characteristics among incoming fourth class midshipmen. In other 
words, this model can be used as a preliminary means of identifying midshipmen for 
academic assistance with the understanding that it will not provide accurate CQPR 
prediction, but will provide an educator with a relative performance rating based upon 
initial entry variables correlated with academic performance. This process could be 
implemented simply by following a few basic steps. 

First, ensure that all initial entry data is collected and formatted correctly in 
accordance with details provided for each variable in the description section of this study. 
Second, filter out any invalid entries. Be aware that if there are missing or invalid entries 
for any included variables of a particular case, the midshipman associated with that case 
would be excluded from the calculation process. In order to avoid the potential problem 


of overlooking a midshipman who may need assistance, it would be valuable to perform a 
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frequency evaluation for each included variable to identify any cases with missing or 
invalid entries. Once these cases are identified, missing or invalid entries can be obtained 
or corrected if possible. Third, compute CQPRprep for each case using Equation (5). In 
the SPSS 9.0 program, this can be performed using the compute function to create new 
variable “CQPRprep*. Fourth, sort the resulting values in ascending order. The resulting 
column of data provides a baseline for relative performance. In this case, CQPRprep 
could be better described as a predicted class ranking rather than a predicted quality point 
rating. The fifth step in the identification process would be to review the calculated data 
implementing subjective criteria such as interviews, recommendations and any other 
desired input to either strengthen or weaken cases for academic assistance consideration. 
Finally, assemble the desired number of midshipmen for academic assistance counseling 
and implement the appropriate program. 

The process described above is by no means perfect. It’s strength lies in its 
flexibility. It would be unrealistic to assume that Equation (5), modeled above, could be 
permanently utilized as a means for identifying midshipmen for academic assistance. 
Changes in culture, academics, and time may alter the relationships between certain 
independent variables and the dependent variable CQPRprep. Therefore it is important to 
maintain a consistent database of information so that historical data may be periodically 
reviewed to develop new equations for modeling CQPRprep. As additional historical 
data is gathered, the relationships between each independent variable and the dependent 
variable may further define themselves and provide a more accurate description of 


academic performance. 
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B. LESSONS LEARNED 


There have been many lessons learned in conducting this research project. Most 
have been encountered during the data collection and analysis process. In attempting to 
establish a valid database of information to include academic grades, LASSI test scores 
and objective entry-level information, it is imperative that organizations keep some form 
of standardization in data storage. In discontinuing the use of the NATS system, the 
Naval Academy was unable to maintain certain pertinent information relative to the 
academic performance of its students. This loss of data prevented a more thorough 
analytical process creating a time gap in valid data obtained. Upon realization of this loss 
of data, the Naval Academy Academic Center has taken steps to ensure that a consistent 
record is being maintained so that further analysis may be conducted if necessary. 

During the cross-tabulation phase of this study, it became readily apparent that 
accurate prediction of academic performance for an individual would be very difficult to 
achieve. By introducing such a large number of objective and subjective variables into 
the prediction equation, the study also introduced a great deal of error into the equation. 
This error could have been attributed to missing variables and/or relationships between 
the variables utilized. The key is to realize that perfect or near-perfect prediction is not a 
realistic goal to achieve. Instead of narrowing goals to achieve perfect prediction and 
actual performance, researchers may find more benefit in broadening goals to achieve 
relative performance. 

Collection and organization of data for the time period observed consumed a great 


deal of time partially due to inconsistencies in valid entries. In order to obtain 
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statistically significant results, it is crucial to spend as much time as necessary during this 


step in order to ensure that research is conducted using valid data. 


C. RECOMMENDATIONS FOR FUTURE RESEARCH 


This study has focused upon the prediction of academic performance using 
objective and subjective criteria for the purpose of assisting the Naval Academy 
Academic Center in targeting midshipmen who would most greatly benefit from their 
services. Another avenue of research could involve assessing the differential in academic 
performance between those midshipmen who participate in the PIP and those of the same 
relative characteristics who did not. Along those same lines, is there a greater benefit to 
intervening early in the academic year (initial identification) or waiting to identify those 
who are already having problems (add-on groups)? 

Still another avenue of this same general topic would be to determine whether 
there is a significant differential in academic performance based upon the time of 
intervention. In other words, is there a significant difference in the level of improvement 
for a student who enters the program at the initial level and one who enters as part of the 
second add on group? 

In a follow-on to this study, a researcher could follow the same steps detailed in 
this thesis to maintain and update the model equation so that any cultural, systemic or 
database refinement changes would be accounted for in future selection processes. 
Maintaining the database may also help refine and possibly improve the prediction 


capability of the model equation. 


oo 


One last research topic could be to break down the candidate multiple (WMMLT) 
into it specific components and attempt to determine whether there are specific subjective 
and objective components of this variable that could be used to predict academic 
performance. Some of these components, such as high school rank, and SAT’s, have 
been researched for this study. However, there are other aspects that could lend to further 
research such as the point system used to grade teacher recommendations, blue and gold 
officer interviews as well as other subjective areas. 

The bottom line is that there is a great deal of research still to be conducted in the 
area of education and educational assistance. Subjective factors may prevent researchers 
from ever fully describing academic performance, but continued review, insight and 
Statistical research may provide even more evidence to help educators to identify those 


who require additional assistance to be successful. 
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